undefined step-generate Cluster name generated: nas-psi-414-02-022017 Is nightly?: false Image repository: devtools_gitops Result path: CI/z-stream-test-pipeline-nas/250220-063841 Result path URL: http://uploader.devtools-gitops--pipeline.svc.cluster.local:8080/CI/z-stream-test-pipeline-nas/250220-063841/ undefined step-create-cluster-dir Directory with cluster info exists, logging in WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. undefined step-clone + '[' false = true ']' + '[' false = true ']' + '[' false = true ']' + CHECKOUT_DIR=/workspace/output/ + '[' false = true ']' + test -z '' + test -z '' + test -z '' + git config --global --add safe.directory /workspace/output + /ko-app/git-init -url=https://gitlab.cee.redhat.com/nas/downstream-CI -revision=remove-plumbing-repo-reference -refspec= -path=/workspace/output/ -sslVerify=false -submodules=true -depth=1 -sparseCheckoutDirectories= {"level":"error","ts":1740033538.0040658,"caller":"git/git.go:53","msg":"Error running git [remote get-url origin]: exit status 2\nerror: No such remote 'origin'\n","stacktrace":"github.com/tektoncd-catalog/git-clone/git-init/git.run\n\t/go/src/github.com/tektoncd-catalog/git-clone/image/git-init/git/git.go:53\ngithub.com/tektoncd-catalog/git-clone/git-init/git.Fetch\n\t/go/src/github.com/tektoncd-catalog/git-clone/image/git-init/git/git.go:109\nmain.main\n\t/go/src/github.com/tektoncd-catalog/git-clone/image/git-init/main.go:52\nruntime.main\n\t/usr/lib/golang/src/runtime/proc.go:271"} {"level":"info","ts":1740033539.100201,"caller":"git/git.go:186","msg":"Successfully cloned https://gitlab.cee.redhat.com/nas/downstream-CI @ d806c3a8467ac6d85bba90e43ca25b96f8353dcd (grafted, HEAD, origin/remove-plumbing-repo-reference) in path /workspace/output/"} {"level":"info","ts":1740033539.1748724,"caller":"git/git.go:225","msg":"Successfully initialized and updated submodules in path /workspace/output/"} + cd /workspace/output/ ++ git rev-parse HEAD + RESULT_SHA=d806c3a8467ac6d85bba90e43ca25b96f8353dcd + EXIT_CODE=0 + '[' 0 '!=' 0 ']' ++ git log -1 --pretty=%ct + RESULT_COMMITTER_DATE=1739946370 + printf %s 1739946370 + printf %s d806c3a8467ac6d85bba90e43ca25b96f8353dcd + printf %s https://gitlab.cee.redhat.com/nas/downstream-CI undefined step-branch-version gitops version latest is Present undefined step-clone + '[' false = true ']' + '[' false = true ']' + '[' false = true ']' + CHECKOUT_DIR=/workspace/output/ + '[' false = true ']' + test -z '' + test -z '' + test -z '' + git config --global --add safe.directory /workspace/output + /ko-app/git-init -url=https://gitlab.cee.redhat.com/gitops/operator-e2e.git -revision=master -refspec= -path=/workspace/output/ -sslVerify=false -submodules=true -depth=1 -sparseCheckoutDirectories= {"level":"error","ts":1740033556.1584382,"caller":"git/git.go:53","msg":"Error running git [remote get-url origin]: exit status 2\nerror: No such remote 'origin'\n","stacktrace":"github.com/tektoncd-catalog/git-clone/git-init/git.run\n\t/go/src/github.com/tektoncd-catalog/git-clone/image/git-init/git/git.go:53\ngithub.com/tektoncd-catalog/git-clone/git-init/git.Fetch\n\t/go/src/github.com/tektoncd-catalog/git-clone/image/git-init/git/git.go:109\nmain.main\n\t/go/src/github.com/tektoncd-catalog/git-clone/image/git-init/main.go:52\nruntime.main\n\t/usr/lib/golang/src/runtime/proc.go:271"} {"level":"info","ts":1740033557.2560656,"caller":"git/git.go:186","msg":"Successfully cloned https://gitlab.cee.redhat.com/gitops/operator-e2e.git @ 90e2ba4bbae467e242eb65e2293ebc32659f9df0 (grafted, HEAD, origin/master) in path /workspace/output/"} {"level":"info","ts":1740033557.3585954,"caller":"git/git.go:225","msg":"Successfully initialized and updated submodules in path /workspace/output/"} + cd /workspace/output/ ++ git rev-parse HEAD + RESULT_SHA=90e2ba4bbae467e242eb65e2293ebc32659f9df0 + EXIT_CODE=0 + '[' 0 '!=' 0 ']' ++ git log -1 --pretty=%ct + RESULT_COMMITTER_DATE=1740032146 + printf %s 1740032146 + printf %s 90e2ba4bbae467e242eb65e2293ebc32659f9df0 + printf %s https://gitlab.cee.redhat.com/gitops/operator-e2e.git undefined step-create-catalog-source Logging in as kubeadmin user WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. NAME DISPLAY TYPE PUBLISHER AGE iib-919669 iib-919669-devtools_gitops grpc GitOps Team 23h pod/iib-919669-9krh9 condition met Operator will be installed from default catalog source iib-919669 passed undefined step-test-argo-rollouts + RESULT=/install-dir/nas-psi-414-02-result.txt Logging in as kubeadmin user + trap update_result EXIT + echo -e 'Logging in as kubeadmin user' ++ cat /install-dir/nas-psi-414-02/auth/kubeadmin-password ++ cat /install-dir/nas-psi-414-02/auth/api-url + oc login -u kubeadmin -p 65KFg-3NTSK-KksWr-gCKbq https://api.nas-psi-414-02.ocp-gitops-qe.com:6443 --insecure-skip-tls-verify=true WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. ++ echo false ++ tr '[:upper:]' '[:lower:]' + [[ false == \f\a\l\s\e ]] $RUN_ARGO_ROLLOUTS_TESTS is set to false, skipping the execution + echo '$RUN_ARGO_ROLLOUTS_TESTS is set to false, skipping the execution' + echo 0 + tee /tekton/results/result 0 + exit 0 + update_result + [[ '' != 0 ]] + echo test-argo-rollouts + tee /install-dir/nas-psi-414-02-result.txt test-argo-rollouts + exit 0 undefined step-run-kuttl-e2e-tests Logging in as kubeadmin user WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. make: Entering directory '/kuttl-e2e/gitops-operator' Running GitOps Operator E2E tests... . ./test.sh all =========================== ==== RUNNING ALL TESTS ==== =========================== Running parallel test suite 2025/02/20 06:40:01 kutt-test config testdirs is overridden with args: [ /kuttl-e2e/gitops-operator/tests/parallel ] === RUN kuttl harness.go:462: starting setup harness.go:252: running tests using configured kubeconfig. I0220 06:40:02.278344 122 request.go:682] Waited for 1.03857547s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s harness.go:275: Successful connection to cluster at: https://api.nas-psi-414-02.ocp-gitops-qe.com:6443 harness.go:360: running tests harness.go:73: going to run test suite with timeout of 720 seconds for each step harness.go:372: testsuite: /kuttl-e2e/gitops-operator/tests/parallel has 89 tests === RUN kuttl/harness === RUN kuttl/harness/1-001_alpha_to_beta_dex_conversion === PAUSE kuttl/harness/1-001_alpha_to_beta_dex_conversion === RUN kuttl/harness/1-002_alpha_to_beta_keycloak_conversion === PAUSE kuttl/harness/1-002_alpha_to_beta_keycloak_conversion === RUN kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion === PAUSE kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion === RUN kuttl/harness/1-003_validate_console_link === PAUSE kuttl/harness/1-003_validate_console_link === RUN kuttl/harness/1-004_beta_to_alpha_conversion === PAUSE kuttl/harness/1-004_beta_to_alpha_conversion === RUN kuttl/harness/1-005_validate_route_tls === PAUSE kuttl/harness/1-005_validate_route_tls === RUN kuttl/harness/1-007_validate_namespace_scoped_install === PAUSE kuttl/harness/1-007_validate_namespace_scoped_install === RUN kuttl/harness/1-007_validate_volume_mounts === PAUSE kuttl/harness/1-007_validate_volume_mounts === RUN kuttl/harness/1-008_validate-custom-argocd-namespace === PAUSE kuttl/harness/1-008_validate-custom-argocd-namespace === RUN kuttl/harness/1-009_validate-manage-other-namespace === PAUSE kuttl/harness/1-009_validate-manage-other-namespace === RUN kuttl/harness/1-012_validate-managed-by-chain === PAUSE kuttl/harness/1-012_validate-managed-by-chain === RUN kuttl/harness/1-019_validate_volume_mounts === PAUSE kuttl/harness/1-019_validate_volume_mounts === RUN kuttl/harness/1-021_validate_rolebindings === PAUSE kuttl/harness/1-021_validate_rolebindings === RUN kuttl/harness/1-023_validate_repo_server_tls === PAUSE kuttl/harness/1-023_validate_repo_server_tls === RUN kuttl/harness/1-025-validate-managed-by-change === PAUSE kuttl/harness/1-025-validate-managed-by-change === RUN kuttl/harness/1-029_validate_tls_secret_no_scale === PAUSE kuttl/harness/1-029_validate_tls_secret_no_scale === RUN kuttl/harness/1-030_validate_reencrypt === PAUSE kuttl/harness/1-030_validate_reencrypt === RUN kuttl/harness/1-032_validate_dynamic_scaling === PAUSE kuttl/harness/1-032_validate_dynamic_scaling === RUN kuttl/harness/1-032_validate_resource_inclusions === PAUSE kuttl/harness/1-032_validate_resource_inclusions === RUN kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount === PAUSE kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount === RUN kuttl/harness/1-033_validate_resource_exclusions === PAUSE kuttl/harness/1-033_validate_resource_exclusions === RUN kuttl/harness/1-034_validate_webhook_notifications === PAUSE kuttl/harness/1-034_validate_webhook_notifications === RUN kuttl/harness/1-036_validate_keycloak_resource_reqs === PAUSE kuttl/harness/1-036_validate_keycloak_resource_reqs === RUN kuttl/harness/1-037_validate_argocd_setting_replicas === PAUSE kuttl/harness/1-037_validate_argocd_setting_replicas === RUN kuttl/harness/1-038_validate_productized_images === PAUSE kuttl/harness/1-038_validate_productized_images === RUN kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm === PAUSE kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm === RUN kuttl/harness/1-042_restricted_pss_compliant === PAUSE kuttl/harness/1-042_restricted_pss_compliant === RUN kuttl/harness/1-043_validate_custom_labels_annotations === PAUSE kuttl/harness/1-043_validate_custom_labels_annotations === RUN kuttl/harness/1-043_validate_log_level_format === PAUSE kuttl/harness/1-043_validate_log_level_format === RUN kuttl/harness/1-044_validate_resource_limit_changes === PAUSE kuttl/harness/1-044_validate_resource_limit_changes === RUN kuttl/harness/1-045_validate_applicationset_image_override === PAUSE kuttl/harness/1-045_validate_applicationset_image_override === RUN kuttl/harness/1-045_validate_repo_exec_timeout === PAUSE kuttl/harness/1-045_validate_repo_exec_timeout === RUN kuttl/harness/1-047_validate_custom_env === PAUSE kuttl/harness/1-047_validate_custom_env === RUN kuttl/harness/1-048_validate_controller_sharding === PAUSE kuttl/harness/1-048_validate_controller_sharding === RUN kuttl/harness/1-049_validate_parallelism_limit === PAUSE kuttl/harness/1-049_validate_parallelism_limit === RUN kuttl/harness/1-050_validate_sso === PAUSE kuttl/harness/1-050_validate_sso === RUN kuttl/harness/1-051-validate_csv_permissions === PAUSE kuttl/harness/1-051-validate_csv_permissions === RUN kuttl/harness/1-052_validate_rolebinding_number === PAUSE kuttl/harness/1-052_validate_rolebinding_number === RUN kuttl/harness/1-053_validate_cluster_admin_rbac === PAUSE kuttl/harness/1-053_validate_cluster_admin_rbac === RUN kuttl/harness/1-054_validate_deploymentconfig === PAUSE kuttl/harness/1-054_validate_deploymentconfig === RUN kuttl/harness/1-055_drop_resource_customizations === PAUSE kuttl/harness/1-055_drop_resource_customizations === RUN kuttl/harness/1-055_validate_notification_controller === PAUSE kuttl/harness/1-055_validate_notification_controller === RUN kuttl/harness/1-057_validate_notifications === PAUSE kuttl/harness/1-057_validate_notifications === RUN kuttl/harness/1-058_validate_prometheus_rule === PAUSE kuttl/harness/1-058_validate_prometheus_rule === RUN kuttl/harness/1-061_validate_resource_tracking_method === PAUSE kuttl/harness/1-061_validate_resource_tracking_method === RUN kuttl/harness/1-062_validate_extra_config === PAUSE kuttl/harness/1-062_validate_extra_config === RUN kuttl/harness/1-063_validate_dex_liveness_probe === PAUSE kuttl/harness/1-063_validate_dex_liveness_probe === RUN kuttl/harness/1-063_validate_statefulset_restart === PAUSE kuttl/harness/1-063_validate_statefulset_restart === RUN kuttl/harness/1-064_validate_security_contexts === PAUSE kuttl/harness/1-064_validate_security_contexts === RUN kuttl/harness/1-065_validate_redis_ha_anti_affinity === PAUSE kuttl/harness/1-065_validate_redis_ha_anti_affinity === RUN kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha === PAUSE kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha === RUN kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha === PAUSE kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha === RUN kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha === PAUSE kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha === RUN kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha === PAUSE kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha === RUN kuttl/harness/1-070_validate_config_management_plugin === PAUSE kuttl/harness/1-070_validate_config_management_plugin === RUN kuttl/harness/1-071_validate_SCC_HA === PAUSE kuttl/harness/1-071_validate_SCC_HA === RUN kuttl/harness/1-072_validate_liveness_probe_removed === PAUSE kuttl/harness/1-072_validate_liveness_probe_removed === RUN kuttl/harness/1-073_validate_rhsso === PAUSE kuttl/harness/1-073_validate_rhsso === RUN kuttl/harness/1-074_validate_terminating_namespace_block === PAUSE kuttl/harness/1-074_validate_terminating_namespace_block === RUN kuttl/harness/1-075_validate_dex_anyuid === PAUSE kuttl/harness/1-075_validate_dex_anyuid === RUN kuttl/harness/1-077_validate_disable_dex_removed === PAUSE kuttl/harness/1-077_validate_disable_dex_removed === RUN kuttl/harness/1-079_validate_vars_for_notificaitons === PAUSE kuttl/harness/1-079_validate_vars_for_notificaitons === RUN kuttl/harness/1-080_validate_regex_support_argocd_rbac === PAUSE kuttl/harness/1-080_validate_regex_support_argocd_rbac === RUN kuttl/harness/1-081_validate_applicationset_deployment === PAUSE kuttl/harness/1-081_validate_applicationset_deployment === RUN kuttl/harness/1-082_validate_node_placement === PAUSE kuttl/harness/1-082_validate_node_placement === RUN kuttl/harness/1-083_validate_kustomize_namereference === PAUSE kuttl/harness/1-083_validate_kustomize_namereference === RUN kuttl/harness/1-083_validate_resource_customization_subkeys === PAUSE kuttl/harness/1-083_validate_resource_customization_subkeys === RUN kuttl/harness/1-084_validate_status_host_ingress === PAUSE kuttl/harness/1-084_validate_status_host_ingress === RUN kuttl/harness/1-086_validate_applicationset_extra_command_args === PAUSE kuttl/harness/1-086_validate_applicationset_extra_command_args === RUN kuttl/harness/1-087_validate_repo_server_settings === PAUSE kuttl/harness/1-087_validate_repo_server_settings === RUN kuttl/harness/1-088_validate_applicationset_add_env_vars === PAUSE kuttl/harness/1-088_validate_applicationset_add_env_vars === RUN kuttl/harness/1-089_validate_extra_repo_commands_args === PAUSE kuttl/harness/1-089_validate_extra_repo_commands_args === RUN kuttl/harness/1-090_validate_permissions === PAUSE kuttl/harness/1-090_validate_permissions === RUN kuttl/harness/1-091_argocd_app_validate-applicationset-status === PAUSE kuttl/harness/1-091_argocd_app_validate-applicationset-status === RUN kuttl/harness/1-092_validate_workload_status_monitoring_alert === PAUSE kuttl/harness/1-092_validate_workload_status_monitoring_alert === RUN kuttl/harness/1-095_validate_dex_clientsecret === PAUSE kuttl/harness/1-095_validate_dex_clientsecret === RUN kuttl/harness/1-096-validate_home_env_argocd_controller === PAUSE kuttl/harness/1-096-validate_home_env_argocd_controller === RUN kuttl/harness/1-098_validate_dex_clientsecret_deprecated === PAUSE kuttl/harness/1-098_validate_dex_clientsecret_deprecated === RUN kuttl/harness/1-099_validate_server_autoscale === PAUSE kuttl/harness/1-099_validate_server_autoscale === RUN kuttl/harness/1-102_validate_handle_terminating_namespaces === PAUSE kuttl/harness/1-102_validate_handle_terminating_namespaces === RUN kuttl/harness/1-103_argocd_alpha_to_beta_conversion === PAUSE kuttl/harness/1-103_argocd_alpha_to_beta_conversion === RUN kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount === PAUSE kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount === RUN kuttl/harness/1-107_host_attribute_sso_provider === PAUSE kuttl/harness/1-107_host_attribute_sso_provider === RUN kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance === PAUSE kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance === RUN kuttl/harness/1-111_validate_appcontroller_initcontainers === PAUSE kuttl/harness/1-111_validate_appcontroller_initcontainers === RUN kuttl/harness/1-112_validate_server_initcontainers === PAUSE kuttl/harness/1-112_validate_server_initcontainers === RUN kuttl/harness/1-113_validate_appcontroller_sidecar === PAUSE kuttl/harness/1-113_validate_appcontroller_sidecar === RUN kuttl/harness/1-114_validate_server_sidecar === PAUSE kuttl/harness/1-114_validate_server_sidecar === RUN kuttl/harness/1-115_validate_controller_extra_command_args === PAUSE kuttl/harness/1-115_validate_controller_extra_command_args === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion === CONT kuttl/harness/1-099_validate_server_autoscale === CONT kuttl/harness/1-083_validate_resource_customization_subkeys === CONT kuttl/harness/1-089_validate_extra_repo_commands_args === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars === CONT kuttl/harness/1-083_validate_resource_customization_subkeys logger.go:42: 06:40:08 | 1-083_validate_resource_customization_subkeys | Creating namespace: kuttl-test-valid-sunbird === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 06:40:08 | 1-001_alpha_to_beta_dex_conversion | Creating namespace: kuttl-test-social-grouper === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 06:40:08 | 1-088_validate_applicationset_add_env_vars | Creating namespace: kuttl-test-decent-emu === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 06:40:08 | 1-089_validate_extra_repo_commands_args | Creating namespace: kuttl-test-relieved-pheasant === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:40:08 | 1-099_validate_server_autoscale | Creating namespace: kuttl-test-electric-dinosaur === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 06:40:08 | 1-001_alpha_to_beta_dex_conversion/1-argocd-dex | starting test step 1-argocd-dex === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 06:40:08 | 1-088_validate_applicationset_add_env_vars/1-basic-argocd-install | starting test step 1-basic-argocd-install === CONT kuttl/harness/1-083_validate_resource_customization_subkeys logger.go:42: 06:40:08 | 1-083_validate_resource_customization_subkeys/1-argocd-with-resource-customization-subkeys | starting test step 1-argocd-with-resource-customization-subkeys === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 06:40:08 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | starting test step 1-basic-argocd-install === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:40:08 | 1-099_validate_server_autoscale/1-install | starting test step 1-install === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 06:40:11 | 1-001_alpha_to_beta_dex_conversion/1-argocd-dex | ArgoCD:kuttl-test-social-grouper/argocd created I0220 06:40:12.315998 122 request.go:682] Waited for 1.032479827s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1beta1?timeout=32s === CONT kuttl/harness/1-083_validate_resource_customization_subkeys logger.go:42: 06:40:16 | 1-083_validate_resource_customization_subkeys/1-argocd-with-resource-customization-subkeys | ArgoCD:kuttl-test-valid-sunbird/example-argocd created === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 06:40:16 | 1-088_validate_applicationset_add_env_vars/1-basic-argocd-install | ArgoCD:kuttl-test-decent-emu/example-argocd created === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 06:40:21 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | Namespace:/test-1-89-repo-extra-command created === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:40:21 | 1-099_validate_server_autoscale/1-install | ArgoCD:kuttl-test-electric-dinosaur/example-argocd created === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 06:40:21 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | ArgoCD:test-1-89-repo-extra-command/example-argocd created === CONT kuttl/harness/1-083_validate_resource_customization_subkeys logger.go:42: 06:40:27 | 1-083_validate_resource_customization_subkeys/1-argocd-with-resource-customization-subkeys | test step completed 1-argocd-with-resource-customization-subkeys logger.go:42: 06:40:27 | 1-083_validate_resource_customization_subkeys | skipping kubernetes event logging logger.go:42: 06:40:27 | 1-083_validate_resource_customization_subkeys | Deleting namespace: kuttl-test-valid-sunbird === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 06:40:33 | 1-087_validate_repo_server_settings | Creating namespace: kuttl-test-live-swine logger.go:42: 06:40:33 | 1-087_validate_repo_server_settings/1-install | starting test step 1-install I0220 06:40:34.638392 122 request.go:682] Waited for 1.031628889s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/packages.operators.coreos.com/v1?timeout=32s logger.go:42: 06:40:36 | 1-087_validate_repo_server_settings/1-install | ArgoCD:kuttl-test-live-swine/example-argocd created === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 06:40:43 | 1-001_alpha_to_beta_dex_conversion/1-argocd-dex | test step completed 1-argocd-dex logger.go:42: 06:40:43 | 1-001_alpha_to_beta_dex_conversion/2-delete | starting test step 2-delete I0220 06:40:44.674216 122 request.go:682] Waited for 1.282570577s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apiextensions.k8s.io/v1?timeout=32s logger.go:42: 06:40:45 | 1-001_alpha_to_beta_dex_conversion/2-delete | test step completed 2-delete logger.go:42: 06:40:45 | 1-001_alpha_to_beta_dex_conversion | skipping kubernetes event logging logger.go:42: 06:40:45 | 1-001_alpha_to_beta_dex_conversion | Deleting namespace: kuttl-test-social-grouper === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 06:40:48 | 1-088_validate_applicationset_add_env_vars/1-basic-argocd-install | test step completed 1-basic-argocd-install logger.go:42: 06:40:48 | 1-088_validate_applicationset_add_env_vars/2-add-env-vars | starting test step 2-add-env-vars logger.go:42: 06:40:51 | 1-088_validate_applicationset_add_env_vars/2-add-env-vars | ArgoCD:kuttl-test-decent-emu/example-argocd updated === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 06:40:52 | 1-086_validate_applicationset_extra_command_args | Creating namespace: kuttl-test-suitable-swan logger.go:42: 06:40:52 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | starting test step 1-basic-argocd-install === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 06:40:54 | 1-088_validate_applicationset_add_env_vars/2-add-env-vars | test step completed 2-add-env-vars logger.go:42: 06:40:54 | 1-088_validate_applicationset_add_env_vars | skipping kubernetes event logging === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:40:54 | 1-099_validate_server_autoscale/1-install | test step completed 1-install logger.go:42: 06:40:54 | 1-099_validate_server_autoscale/2-enable-server-autoscale | starting test step 2-enable-server-autoscale === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 06:40:54 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | test step completed 1-basic-argocd-install logger.go:42: 06:40:54 | 1-089_validate_extra_repo_commands_args/2-add-extraRepoCommandArgs | starting test step 2-add-extraRepoCommandArgs === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 06:40:54 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | Namespace:/test-1-86-appsets-extra-command created === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 06:40:54 | 1-088_validate_applicationset_add_env_vars | Deleting namespace: kuttl-test-decent-emu === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 06:40:54 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | ArgoCD:test-1-86-appsets-extra-command/example-argocd created I0220 06:40:55.760106 122 request.go:682] Waited for 1.029557328s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1beta1?timeout=32s === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:40:57 | 1-099_validate_server_autoscale/2-enable-server-autoscale | ArgoCD:kuttl-test-electric-dinosaur/example-argocd updated logger.go:42: 06:40:57 | 1-099_validate_server_autoscale/2-enable-server-autoscale | test step completed 2-enable-server-autoscale logger.go:42: 06:40:57 | 1-099_validate_server_autoscale/3-check-server-replicas | starting test step 3-check-server-replicas logger.go:42: 06:40:59 | 1-099_validate_server_autoscale/3-check-server-replicas | running command: [sh -c sleep 45] === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 06:40:59 | 1-089_validate_extra_repo_commands_args/2-add-extraRepoCommandArgs | ArgoCD:test-1-89-repo-extra-command/example-argocd updated logger.go:42: 06:41:00 | 1-089_validate_extra_repo_commands_args/2-add-extraRepoCommandArgs | test step completed 2-add-extraRepoCommandArgs logger.go:42: 06:41:00 | 1-089_validate_extra_repo_commands_args/99-delete | starting test step 99-delete === CONT kuttl/harness/1-084_validate_status_host_ingress logger.go:42: 06:41:00 | 1-084_validate_status_host_ingress | Creating namespace: kuttl-test-quick-bluebird logger.go:42: 06:41:00 | 1-084_validate_status_host_ingress/1-argocd-ingress | starting test step 1-argocd-ingress logger.go:42: 06:41:03 | 1-084_validate_status_host_ingress/1-argocd-ingress | ArgoCD:kuttl-test-quick-bluebird/example-argocd created === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 06:41:08 | 1-087_validate_repo_server_settings/1-install | test step completed 1-install logger.go:42: 06:41:08 | 1-087_validate_repo_server_settings/2-modify-argocd | starting test step 2-modify-argocd I0220 06:41:09.633926 122 request.go:682] Waited for 1.031723704s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1alpha5?timeout=32s logger.go:42: 06:41:11 | 1-087_validate_repo_server_settings/2-modify-argocd | ArgoCD:kuttl-test-live-swine/example-argocd updated logger.go:42: 06:41:12 | 1-087_validate_repo_server_settings/2-modify-argocd | test step completed 2-modify-argocd logger.go:42: 06:41:12 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | starting test step 4-modify-serviceaccount logger.go:42: 06:41:14 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | ServiceAccount:kuttl-test-live-swine/modified-default created logger.go:42: 06:41:14 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | ArgoCD:kuttl-test-live-swine/example-argocd updated === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 06:41:17 | 1-089_validate_extra_repo_commands_args/99-delete | test step completed 99-delete logger.go:42: 06:41:17 | 1-089_validate_extra_repo_commands_args | skipping kubernetes event logging logger.go:42: 06:41:17 | 1-089_validate_extra_repo_commands_args | Deleting namespace: kuttl-test-relieved-pheasant === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 06:41:17 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | test step completed 1-basic-argocd-install logger.go:42: 06:41:17 | 1-086_validate_applicationset_extra_command_args/2-add-extraCommandArgs | starting test step 2-add-extraCommandArgs === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 06:41:17 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | test step completed 4-modify-serviceaccount logger.go:42: 06:41:17 | 1-087_validate_repo_server_settings/5-reset-repo-settings | starting test step 5-reset-repo-settings I0220 06:41:19.635635 122 request.go:682] Waited for 2.381050771s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 06:41:19 | 1-086_validate_applicationset_extra_command_args/2-add-extraCommandArgs | ArgoCD:test-1-86-appsets-extra-command/example-argocd updated logger.go:42: 06:41:22 | 1-086_validate_applicationset_extra_command_args/2-add-extraCommandArgs | test step completed 2-add-extraCommandArgs logger.go:42: 06:41:22 | 1-086_validate_applicationset_extra_command_args/99-delete | starting test step 99-delete === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 06:41:22 | 1-087_validate_repo_server_settings/5-reset-repo-settings | ArgoCD:kuttl-test-live-swine/example-argocd updated === CONT kuttl/harness/1-090_validate_permissions logger.go:42: 06:41:23 | 1-090_validate_permissions | Creating namespace: kuttl-test-pleasing-stag logger.go:42: 06:41:23 | 1-090_validate_permissions/1- | starting test step 1- === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 06:41:25 | 1-087_validate_repo_server_settings/5-reset-repo-settings | test step completed 5-reset-repo-settings logger.go:42: 06:41:25 | 1-087_validate_repo_server_settings | skipping kubernetes event logging === CONT kuttl/harness/1-090_validate_permissions logger.go:42: 06:41:25 | 1-090_validate_permissions/1- | test step completed 1- logger.go:42: 06:41:25 | 1-090_validate_permissions/2- | starting test step 2- === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 06:41:25 | 1-087_validate_repo_server_settings | Deleting namespace: kuttl-test-live-swine === CONT kuttl/harness/1-090_validate_permissions logger.go:42: 06:41:28 | 1-090_validate_permissions/2- | test step completed 2- logger.go:42: 06:41:28 | 1-090_validate_permissions | skipping kubernetes event logging logger.go:42: 06:41:28 | 1-090_validate_permissions | Deleting namespace: kuttl-test-pleasing-stag === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 06:41:31 | 1-098_validate_dex_clientsecret_deprecated | Creating namespace: kuttl-test-amazed-opossum logger.go:42: 06:41:32 | 1-098_validate_dex_clientsecret_deprecated/1-install | starting test step 1-install I0220 06:41:33.053166 122 request.go:682] Waited for 1.030695012s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1?timeout=32s === CONT kuttl/harness/1-111_validate_appcontroller_initcontainers logger.go:42: 06:41:34 | 1-111_validate_appcontroller_initcontainers | Creating namespace: kuttl-test-prime-ringtail logger.go:42: 06:41:34 | 1-111_validate_appcontroller_initcontainers/1-install | starting test step 1-install === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 06:41:34 | 1-098_validate_dex_clientsecret_deprecated/1-install | ArgoCD:kuttl-test-amazed-opossum/example-argocd created === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 06:41:37 | 1-086_validate_applicationset_extra_command_args/99-delete | test step completed 99-delete logger.go:42: 06:41:37 | 1-086_validate_applicationset_extra_command_args | skipping kubernetes event logging logger.go:42: 06:41:37 | 1-086_validate_applicationset_extra_command_args | Deleting namespace: kuttl-test-suitable-swan === CONT kuttl/harness/1-084_validate_status_host_ingress logger.go:42: 06:41:39 | 1-084_validate_status_host_ingress/1-argocd-ingress | test step completed 1-argocd-ingress logger.go:42: 06:41:39 | 1-084_validate_status_host_ingress/2- | starting test step 2- === CONT kuttl/harness/1-111_validate_appcontroller_initcontainers logger.go:42: 06:41:39 | 1-111_validate_appcontroller_initcontainers/1-install | ArgoCD:kuttl-test-prime-ringtail/argocd created === CONT kuttl/harness/1-084_validate_status_host_ingress logger.go:42: 06:41:42 | 1-084_validate_status_host_ingress/2- | test step completed 2- logger.go:42: 06:41:42 | 1-084_validate_status_host_ingress | skipping kubernetes event logging === CONT kuttl/harness/1-111_validate_appcontroller_initcontainers logger.go:42: 06:41:42 | 1-111_validate_appcontroller_initcontainers/1-install | test step completed 1-install logger.go:42: 06:41:42 | 1-111_validate_appcontroller_initcontainers | skipping kubernetes event logging === CONT kuttl/harness/1-084_validate_status_host_ingress logger.go:42: 06:41:42 | 1-084_validate_status_host_ingress | Deleting namespace: kuttl-test-quick-bluebird === CONT kuttl/harness/1-111_validate_appcontroller_initcontainers logger.go:42: 06:41:42 | 1-111_validate_appcontroller_initcontainers | Deleting namespace: kuttl-test-prime-ringtail === CONT kuttl/harness/1-096-validate_home_env_argocd_controller logger.go:42: 06:41:43 | 1-096-validate_home_env_argocd_controller | Creating namespace: kuttl-test-good-bird logger.go:42: 06:41:43 | 1-096-validate_home_env_argocd_controller/1- | starting test step 1- I0220 06:41:44.166862 122 request.go:682] Waited for 1.032759717s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:41:44 | 1-099_validate_server_autoscale/3-check-server-replicas | running command: [sh -c set -e serverReplicas=$(kubectl get -n $NAMESPACE deployment/example-argocd-server -o jsonpath='{.spec.replicas}') if [ "$serverReplicas" -ge 4 ] && [ "$serverReplicas" -le 7 ]; then echo "server replica count matches expectation" exit 0 fi exit 1 done ] logger.go:42: 06:41:45 | 1-099_validate_server_autoscale/3-check-server-replicas | server replica count matches expectation === CONT kuttl/harness/1-096-validate_home_env_argocd_controller logger.go:42: 06:41:45 | 1-096-validate_home_env_argocd_controller/1- | test step completed 1- logger.go:42: 06:41:45 | 1-096-validate_home_env_argocd_controller | skipping kubernetes event logging logger.go:42: 06:41:45 | 1-096-validate_home_env_argocd_controller | Deleting namespace: kuttl-test-good-bird === CONT kuttl/harness/1-115_validate_controller_extra_command_args === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:41:48 | 1-099_validate_server_autoscale/3-check-server-replicas | test step completed 3-check-server-replicas logger.go:42: 06:41:48 | 1-099_validate_server_autoscale/4-update-hpa | starting test step 4-update-hpa === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 06:41:50 | 1-095_validate_dex_clientsecret | Creating namespace: kuttl-test-mint-swan === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:41:50 | 1-115_validate_controller_extra_command_args | Creating namespace: kuttl-test-fast-chicken === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 06:41:50 | 1-095_validate_dex_clientsecret/1-install | starting test step 1-install === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:41:50 | 1-115_validate_controller_extra_command_args/1-basic-argocd-install | starting test step 1-basic-argocd-install === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:41:50 | 1-099_validate_server_autoscale/4-update-hpa | ArgoCD:kuttl-test-electric-dinosaur/example-argocd updated === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 06:41:53 | 1-098_validate_dex_clientsecret_deprecated/1-install | test step completed 1-install logger.go:42: 06:41:53 | 1-098_validate_dex_clientsecret_deprecated/2-verify-clientsecret | starting test step 2-verify-clientsecret === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 06:41:53 | 1-095_validate_dex_clientsecret/1-install | ArgoCD:kuttl-test-mint-swan/example-argocd created I0220 06:41:54.229577 122 request.go:682] Waited for 1.028387703s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:41:55 | 1-092_validate_workload_status_monitoring_alert | Creating namespace: kuttl-test-giving-seasnail === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 06:41:55 | 1-098_validate_dex_clientsecret_deprecated/2-verify-clientsecret | running command: [sh -c # This test validates the Dex Client Secret copied by the operator from dex serviceaccount token secret in to argocd-secret. # To verify the behavior we should first get the token secret name of the dex service account. secret=$(oc get -n $NAMESPACE sa example-argocd-argocd-dex-server -o json | jq -r '.secrets' | grep token | sed 's/ "name": "//g' | sed 's/"//g') # Extract the clientSecret expectedClientSecret=$(oc get secret $secret -n $NAMESPACE -o json | jq -r '.data.token') # actualClientSecret is the value of the secret in argocd-secret where argocd-operator should copy the secret from actualClientSecret=$(oc get secret argocd-secret -o json -n $NAMESPACE | jq -r '.data."oidc.dex.clientSecret"') # Verify if [[ ${expectedClientSecret} != ${actualClientSecret} ]]; then echo "Error: Dex Client Secret for OIDC is not valid" exit 1 fi ] === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:41:55 | 1-092_validate_workload_status_monitoring_alert/1-install | starting test step 1-install === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:41:55 | 1-099_validate_server_autoscale/4-update-hpa | test step completed 4-update-hpa logger.go:42: 06:41:55 | 1-099_validate_server_autoscale | skipping kubernetes event logging === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:41:55 | 1-115_validate_controller_extra_command_args/1-basic-argocd-install | Namespace:/test-1-38-controller-extra-command created === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 06:41:55 | 1-099_validate_server_autoscale | Deleting namespace: kuttl-test-electric-dinosaur === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:41:55 | 1-115_validate_controller_extra_command_args/1-basic-argocd-install | ArgoCD:test-1-38-controller-extra-command/example-argocd created === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:41:58 | 1-092_validate_workload_status_monitoring_alert/1-install | ArgoCD:kuttl-test-giving-seasnail/argocd created === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 06:42:00 | 1-098_validate_dex_clientsecret_deprecated/2-verify-clientsecret | test step completed 2-verify-clientsecret logger.go:42: 06:42:00 | 1-098_validate_dex_clientsecret_deprecated | skipping kubernetes event logging logger.go:42: 06:42:00 | 1-098_validate_dex_clientsecret_deprecated | Deleting namespace: kuttl-test-amazed-opossum === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 06:42:02 | 1-091_argocd_app_validate-applicationset-status | Creating namespace: kuttl-test-poetic-liger logger.go:42: 06:42:02 | 1-091_argocd_app_validate-applicationset-status/1-install | starting test step 1-install I0220 06:42:04.239774 122 request.go:682] Waited for 1.680130349s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 06:42:05 | 1-091_argocd_app_validate-applicationset-status/1-install | ArgoCD:kuttl-test-poetic-liger/example-argocd created logger.go:42: 06:42:06 | 1-091_argocd_app_validate-applicationset-status/1-install | test step completed 1-install logger.go:42: 06:42:06 | 1-091_argocd_app_validate-applicationset-status/2-enable-appset | starting test step 2-enable-appset === CONT kuttl/harness/1-114_validate_server_sidecar logger.go:42: 06:42:08 | 1-114_validate_server_sidecar | Creating namespace: kuttl-test-wealthy-baboon logger.go:42: 06:42:08 | 1-114_validate_server_sidecar/1-install | starting test step 1-install === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 06:42:08 | 1-091_argocd_app_validate-applicationset-status/2-enable-appset | ArgoCD:kuttl-test-poetic-liger/example-argocd updated === CONT kuttl/harness/1-114_validate_server_sidecar logger.go:42: 06:42:11 | 1-114_validate_server_sidecar/1-install | ArgoCD:kuttl-test-wealthy-baboon/argocd created === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 06:42:12 | 1-091_argocd_app_validate-applicationset-status/2-enable-appset | test step completed 2-enable-appset logger.go:42: 06:42:12 | 1-091_argocd_app_validate-applicationset-status/3-change-appset-image | starting test step 3-change-appset-image I0220 06:42:14.289315 122 request.go:682] Waited for 2.081459609s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s === CONT kuttl/harness/1-114_validate_server_sidecar logger.go:42: 06:42:14 | 1-114_validate_server_sidecar/1-install | test step completed 1-install logger.go:42: 06:42:14 | 1-114_validate_server_sidecar | skipping kubernetes event logging logger.go:42: 06:42:14 | 1-114_validate_server_sidecar | Deleting namespace: kuttl-test-wealthy-baboon === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 06:42:14 | 1-091_argocd_app_validate-applicationset-status/3-change-appset-image | ArgoCD:kuttl-test-poetic-liger/example-argocd updated logger.go:42: 06:42:14 | 1-091_argocd_app_validate-applicationset-status/3-change-appset-image | test step completed 3-change-appset-image logger.go:42: 06:42:14 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | starting test step 4-delete-exisiting-deployment logger.go:42: 06:42:14 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | running command: [kubectl delete deployment example-argocd-applicationset-controller -n kuttl-test-poetic-liger] logger.go:42: 06:42:15 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | deployment.apps "example-argocd-applicationset-controller" deleted logger.go:42: 06:42:17 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | test step completed 4-delete-exisiting-deployment logger.go:42: 06:42:17 | 1-091_argocd_app_validate-applicationset-status | skipping kubernetes event logging logger.go:42: 06:42:17 | 1-091_argocd_app_validate-applicationset-status | Deleting namespace: kuttl-test-poetic-liger === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 06:42:20 | 1-104_validate_applicationset_tls_scm_volume_mount | Creating namespace: kuttl-test-evolving-goblin logger.go:42: 06:42:20 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | starting test step 1-install logger.go:42: 06:42:23 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | Namespace:/test-1-104-appsets-scm-tls-mount created logger.go:42: 06:42:23 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | ConfigMap:test-1-104-appsets-scm-tls-mount/test-1-104-appsets-scm-tls-cm created logger.go:42: 06:42:23 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | ArgoCD:test-1-104-appsets-scm-tls-mount/argocd created === CONT kuttl/harness/1-113_validate_appcontroller_sidecar logger.go:42: 06:42:24 | 1-113_validate_appcontroller_sidecar | Creating namespace: kuttl-test-adjusted-monitor logger.go:42: 06:42:24 | 1-113_validate_appcontroller_sidecar/1-install | starting test step 1-install I0220 06:42:25.115564 122 request.go:682] Waited for 1.033229359s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/tuned.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 06:42:26 | 1-095_validate_dex_clientsecret/1-install | test step completed 1-install logger.go:42: 06:42:26 | 1-095_validate_dex_clientsecret/2-check-clientSecret-cm | starting test step 2-check-clientSecret-cm logger.go:42: 06:42:26 | 1-095_validate_dex_clientsecret/2-check-clientSecret-cm | running command: [sh -c clientSecret=$(oc get configmap argocd-cm -n $NAMESPACE -o jsonpath='{.data.dex\.config}' | grep clientSecret | awk -F' ' '{print $2}') if test "$clientSecret" != "\$oidc.dex.clientSecret"; then echo "Client secret exposed!" exit 1 fi exit 0 ] === CONT kuttl/harness/1-113_validate_appcontroller_sidecar logger.go:42: 06:42:26 | 1-113_validate_appcontroller_sidecar/1-install | ArgoCD:kuttl-test-adjusted-monitor/argocd created === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 06:42:29 | 1-095_validate_dex_clientsecret/2-check-clientSecret-cm | test step completed 2-check-clientSecret-cm logger.go:42: 06:42:29 | 1-095_validate_dex_clientsecret/3-verify-clientsecret-secret | starting test step 3-verify-clientsecret-secret logger.go:42: 06:42:29 | 1-095_validate_dex_clientsecret/3-verify-clientsecret-secret | running command: [sh -c # This test validates the Dex Client Secret copied by the operator from dex serviceaccount token secret in to argocd-secret. # To verify the behavior we should first get the token secret name of the dex service account. secret=$(oc get -n $NAMESPACE sa example-argocd-argocd-dex-server -o json | jq -r '.secrets[] | select(.name | contains("token")) | .name') # Extract the clientSecret expectedClientSecret=$(oc get secret $secret -n $NAMESPACE -o json | jq -r '.data.token') # actualClientSecret is the value of the secret in argocd-secret where argocd-operator should copy the secret from actualClientSecret=$(oc get secret argocd-secret -o json -n $NAMESPACE | jq -r '.data."oidc.dex.clientSecret"') # Verify if [[ $expectedClientSecret != $actualClientSecret ]]; then echo "Error: Dex Client Secret for OIDC is not valid" exit 1 fi ] === CONT kuttl/harness/1-113_validate_appcontroller_sidecar logger.go:42: 06:42:29 | 1-113_validate_appcontroller_sidecar/1-install | test step completed 1-install logger.go:42: 06:42:29 | 1-113_validate_appcontroller_sidecar | skipping kubernetes event logging === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:29 | 1-115_validate_controller_extra_command_args/1-basic-argocd-install | test step completed 1-basic-argocd-install logger.go:42: 06:42:29 | 1-115_validate_controller_extra_command_args/2-add-extraCommandArgs | starting test step 2-add-extraCommandArgs === CONT kuttl/harness/1-113_validate_appcontroller_sidecar logger.go:42: 06:42:29 | 1-113_validate_appcontroller_sidecar | Deleting namespace: kuttl-test-adjusted-monitor === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:32 | 1-115_validate_controller_extra_command_args/2-add-extraCommandArgs | ArgoCD:test-1-38-controller-extra-command/example-argocd updated === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 06:42:34 | 1-095_validate_dex_clientsecret/3-verify-clientsecret-secret | test step completed 3-verify-clientsecret-secret logger.go:42: 06:42:34 | 1-095_validate_dex_clientsecret | skipping kubernetes event logging === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:42:34 | 1-092_validate_workload_status_monitoring_alert/1-install | test step completed 1-install === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:34 | 1-115_validate_controller_extra_command_args/2-add-extraCommandArgs | test step completed 2-add-extraCommandArgs === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:42:34 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | starting test step 2-verify-alert === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:34 | 1-115_validate_controller_extra_command_args/3-remove-extraCommandArgs | starting test step 3-remove-extraCommandArgs logger.go:42: 06:42:34 | 1-115_validate_controller_extra_command_args/3-remove-extraCommandArgs | running command: [sh -c oc patch -n test-1-38-controller-extra-command argocds/example-argocd --type=json --patch '[{"op": "remove", "path": "/spec/controller/extraCommandArgs"}]' ] === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:42:34 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | running command: [sh -c sleep 7m] === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 06:42:34 | 1-095_validate_dex_clientsecret | Deleting namespace: kuttl-test-mint-swan === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:34 | 1-115_validate_controller_extra_command_args/3-remove-extraCommandArgs | argocd.argoproj.io/example-argocd patched === CONT kuttl/harness/1-107_host_attribute_sso_provider I0220 06:42:36.054160 122 request.go:682] Waited for 1.033396103s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apps.openshift.io/v1?timeout=32s logger.go:42: 06:42:37 | 1-107_host_attribute_sso_provider | Creating namespace: kuttl-test-intimate-emu === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:37 | 1-115_validate_controller_extra_command_args/3-remove-extraCommandArgs | test step completed 3-remove-extraCommandArgs logger.go:42: 06:42:37 | 1-115_validate_controller_extra_command_args/4-override-cli-argument | starting test step 4-override-cli-argument === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 06:42:37 | 1-107_host_attribute_sso_provider/1-install | starting test step 1-install === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:40 | 1-115_validate_controller_extra_command_args/4-override-cli-argument | ArgoCD:test-1-38-controller-extra-command/example-argocd updated === CONT kuttl/harness/1-112_validate_server_initcontainers logger.go:42: 06:42:42 | 1-112_validate_server_initcontainers | Creating namespace: kuttl-test-noble-ostrich logger.go:42: 06:42:42 | 1-112_validate_server_initcontainers/1-install | starting test step 1-install === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:42 | 1-115_validate_controller_extra_command_args/4-override-cli-argument | test step completed 4-override-cli-argument logger.go:42: 06:42:42 | 1-115_validate_controller_extra_command_args/99-delete | starting test step 99-delete === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 06:42:42 | 1-107_host_attribute_sso_provider/1-install | ArgoCD:kuttl-test-intimate-emu/example-argocd-keycloak created === CONT kuttl/harness/1-112_validate_server_initcontainers logger.go:42: 06:42:45 | 1-112_validate_server_initcontainers/1-install | ArgoCD:kuttl-test-noble-ostrich/argocd created logger.go:42: 06:42:46 | 1-112_validate_server_initcontainers/1-install | test step completed 1-install logger.go:42: 06:42:46 | 1-112_validate_server_initcontainers | skipping kubernetes event logging logger.go:42: 06:42:46 | 1-112_validate_server_initcontainers | Deleting namespace: kuttl-test-noble-ostrich === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 06:42:52 | 1-048_validate_controller_sharding | Creating namespace: kuttl-test-master-coyote logger.go:42: 06:42:52 | 1-048_validate_controller_sharding/1-install | starting test step 1-install I0220 06:42:53.084519 122 request.go:682] Waited for 1.033011202s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 06:42:54 | 1-048_validate_controller_sharding/1-install | ArgoCD:kuttl-test-master-coyote/argocd created === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 06:42:55 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | test step completed 1-install logger.go:42: 06:42:55 | 1-104_validate_applicationset_tls_scm_volume_mount/99-delete | starting test step 99-delete === CONT kuttl/harness/1-115_validate_controller_extra_command_args logger.go:42: 06:42:59 | 1-115_validate_controller_extra_command_args/99-delete | test step completed 99-delete logger.go:42: 06:42:59 | 1-115_validate_controller_extra_command_args | skipping kubernetes event logging logger.go:42: 06:42:59 | 1-115_validate_controller_extra_command_args | Deleting namespace: kuttl-test-fast-chicken === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 06:43:05 | 1-083_validate_kustomize_namereference | Creating namespace: kuttl-test-thankful-alien logger.go:42: 06:43:05 | 1-083_validate_kustomize_namereference/1-install | starting test step 1-install I0220 06:43:06.756442 122 request.go:682] Waited for 1.028721756s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s logger.go:42: 06:43:08 | 1-083_validate_kustomize_namereference/1-install | Namespace:/namespace-gitops-2038 created logger.go:42: 06:43:08 | 1-083_validate_kustomize_namereference/1-install | ArgoCD:namespace-gitops-2038/argocd created === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 06:43:14 | 1-107_host_attribute_sso_provider/1-install | test step completed 1-install logger.go:42: 06:43:14 | 1-107_host_attribute_sso_provider | skipping kubernetes event logging logger.go:42: 06:43:14 | 1-107_host_attribute_sso_provider | Deleting namespace: kuttl-test-intimate-emu === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 06:43:20 | 1-083_validate_kustomize_namereference/1-install | test step completed 1-install logger.go:42: 06:43:20 | 1-083_validate_kustomize_namereference/2-install-kustomize-app | starting test step 2-install-kustomize-app I0220 06:43:21.527698 122 request.go:682] Waited for 1.03230295s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s logger.go:42: 06:43:23 | 1-083_validate_kustomize_namereference/2-install-kustomize-app | Application:namespace-gitops-2038/app-kustomize created === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 06:43:26 | 1-048_validate_controller_sharding/1-install | test step completed 1-install logger.go:42: 06:43:26 | 1-048_validate_controller_sharding/2-change-sharding | starting test step 2-change-sharding === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 06:43:28 | 1-083_validate_kustomize_namereference/2-install-kustomize-app | test step completed 2-install-kustomize-app logger.go:42: 06:43:28 | 1-083_validate_kustomize_namereference/99-delete | starting test step 99-delete === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 06:43:28 | 1-048_validate_controller_sharding/2-change-sharding | ArgoCD:kuttl-test-master-coyote/argocd updated I0220 06:43:41.466392 122 request.go:682] Waited for 1.032727339s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta3?timeout=32s === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 06:43:42 | 1-083_validate_kustomize_namereference/99-delete | test step completed 99-delete logger.go:42: 06:43:42 | 1-083_validate_kustomize_namereference | skipping kubernetes event logging logger.go:42: 06:43:42 | 1-083_validate_kustomize_namereference | Deleting namespace: kuttl-test-thankful-alien === CONT kuttl/harness/1-065_validate_redis_ha_anti_affinity logger.go:42: 06:43:48 | 1-065_validate_redis_ha_anti_affinity | Creating namespace: kuttl-test-internal-beetle logger.go:42: 06:43:48 | 1-065_validate_redis_ha_anti_affinity/1-install | starting test step 1-install logger.go:42: 06:43:51 | 1-065_validate_redis_ha_anti_affinity/1-install | ArgoCD:kuttl-test-internal-beetle/argocd created logger.go:42: 06:43:51 | 1-065_validate_redis_ha_anti_affinity/1-install | test step completed 1-install logger.go:42: 06:43:51 | 1-065_validate_redis_ha_anti_affinity/2- | starting test step 2- I0220 06:43:52.513438 122 request.go:682] Waited for 1.031518412s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 06:44:01 | 1-048_validate_controller_sharding/2-change-sharding | test step completed 2-change-sharding logger.go:42: 06:44:01 | 1-048_validate_controller_sharding/3-check-env | starting test step 3-check-env logger.go:42: 06:44:01 | 1-048_validate_controller_sharding/3-check-env | running command: [sh -c replicas=$(oc get -n $NAMESPACE statefulset argocd-application-controller -o json \ | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="ARGOCD_CONTROLLER_REPLICAS").value') if test "$replicas" != "3"; then echo "Environment ARGOCD_CONTROLLER_REPLICAS not correct. Should '3', is '$replicas'" exit 1 fi ] I0220 06:44:02.867617 122 request.go:682] Waited for 1.032370159s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/packages.operators.coreos.com/v1?timeout=32s logger.go:42: 06:44:04 | 1-048_validate_controller_sharding/3-check-env | test step completed 3-check-env logger.go:42: 06:44:04 | 1-048_validate_controller_sharding/4-change-sharding | starting test step 4-change-sharding logger.go:42: 06:44:06 | 1-048_validate_controller_sharding/4-change-sharding | ArgoCD:kuttl-test-master-coyote/argocd updated logger.go:42: 06:44:09 | 1-048_validate_controller_sharding/4-change-sharding | test step completed 4-change-sharding logger.go:42: 06:44:09 | 1-048_validate_controller_sharding/5-check-env | starting test step 5-check-env logger.go:42: 06:44:09 | 1-048_validate_controller_sharding/5-check-env | running command: [sh -c # we do expect error from jq here if env is not set at all replicas=$(oc get -n $NAMESPACE statefulset argocd-application-controller -o json \ | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="ARGOCD_CONTROLLER_REPLICASE").value') if test "$replicas" != ""; then echo "Environment ARGOCD_CONTROLLER_REPLICAS not correct. Should '', is '$replicas'" exit 1 fi ] I0220 06:44:12.886495 122 request.go:682] Waited for 2.43242671s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/template.openshift.io/v1?timeout=32s logger.go:42: 06:44:12 | 1-048_validate_controller_sharding/5-check-env | test step completed 5-check-env logger.go:42: 06:44:12 | 1-048_validate_controller_sharding | skipping kubernetes event logging logger.go:42: 06:44:12 | 1-048_validate_controller_sharding | Deleting namespace: kuttl-test-master-coyote === CONT kuttl/harness/1-064_validate_security_contexts logger.go:42: 06:44:19 | 1-064_validate_security_contexts | Creating namespace: kuttl-test-champion-turkey logger.go:42: 06:44:19 | 1-064_validate_security_contexts/1-install | starting test step 1-install logger.go:42: 06:44:21 | 1-064_validate_security_contexts/1-install | ArgoCD:kuttl-test-champion-turkey/argocd created logger.go:42: 06:44:22 | 1-064_validate_security_contexts/1-install | test step completed 1-install logger.go:42: 06:44:22 | 1-064_validate_security_contexts/2-check-controller-manager | starting test step 2-check-controller-manager logger.go:42: 06:44:22 | 1-064_validate_security_contexts/2-check-controller-manager | running command: [sh -c set -eo pipefail ocp_major_version=$(oc version -o json | jq -r '.openshiftVersion' | cut -d'.' -f 1) ocp_minor_version=$(oc version -o json | jq -r '.openshiftVersion' | cut -d'.' -f 2) security_context=$(oc get deployment/openshift-gitops-operator-controller-manager -n openshift-operators -o jsonpath='{.spec.template.spec.containers[].securityContext}') expected_security_context='{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsNonRoot":true}' if [ "${ocp_major_version}" -eq 4 ] && [ "${ocp_minor_version}" -ge 12 ]; then if [ "${security_context}" = "${expected_security_context}" ]; then exit 0 else echo "Error: deployment/openshift-gitops-operator-controller-manager container's security context is wrong" exit 1 fi else echo "Info: This test is only applicable to OCP version 4.12 and higher." exit 0 fi ] logger.go:42: 06:44:24 | 1-064_validate_security_contexts/2-check-controller-manager | Error from server (NotFound): deployments.apps "openshift-gitops-operator-controller-manager" not found I0220 06:44:25.114476 122 request.go:682] Waited for 1.032827362s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/tuned.openshift.io/v1?timeout=32s case.go:364: failed in step 2-check-controller-manager case.go:366: exit status 1 logger.go:42: 06:44:26 | 1-064_validate_security_contexts | skipping kubernetes event logging logger.go:42: 06:44:26 | 1-064_validate_security_contexts | Deleting namespace: kuttl-test-champion-turkey === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 06:44:32 | 1-082_validate_node_placement | Creating namespace: kuttl-test-divine-duck logger.go:42: 06:44:32 | 1-082_validate_node_placement/1-default-node-selector | starting test step 1-default-node-selector I0220 06:44:35.151162 122 request.go:682] Waited for 2.383311484s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/rbac.authorization.k8s.io/v1?timeout=32s logger.go:42: 06:44:35 | 1-082_validate_node_placement/1-default-node-selector | ArgoCD:kuttl-test-divine-duck/example-argocd created logger.go:42: 06:44:36 | 1-082_validate_node_placement/1-default-node-selector | test step completed 1-default-node-selector logger.go:42: 06:44:36 | 1-082_validate_node_placement/2-custom-node-placement | starting test step 2-custom-node-placement logger.go:42: 06:44:38 | 1-082_validate_node_placement/2-custom-node-placement | ArgoCD:kuttl-test-divine-duck/example-argocd updated logger.go:42: 06:44:40 | 1-082_validate_node_placement/2-custom-node-placement | test step completed 2-custom-node-placement logger.go:42: 06:44:40 | 1-082_validate_node_placement/3-node-placement-with-toleration | starting test step 3-node-placement-with-toleration logger.go:42: 06:44:42 | 1-082_validate_node_placement/3-node-placement-with-toleration | ArgoCD:kuttl-test-divine-duck/example-argocd updated === CONT kuttl/harness/1-063_validate_statefulset_restart logger.go:42: 06:44:43 | 1-063_validate_statefulset_restart | Creating namespace: kuttl-test-amazed-ox logger.go:42: 06:44:43 | 1-063_validate_statefulset_restart/1-install | starting test step 1-install I0220 06:44:45.184964 122 request.go:682] Waited for 1.381477519s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 06:44:46 | 1-063_validate_statefulset_restart/1-install | ArgoCD:kuttl-test-amazed-ox/example-argocd created === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 06:44:46 | 1-082_validate_node_placement/3-node-placement-with-toleration | test step completed 3-node-placement-with-toleration logger.go:42: 06:44:46 | 1-082_validate_node_placement | skipping kubernetes event logging logger.go:42: 06:44:46 | 1-082_validate_node_placement | Deleting namespace: kuttl-test-divine-duck === CONT kuttl/harness/1-081_validate_applicationset_deployment logger.go:42: 06:44:52 | 1-081_validate_applicationset_deployment | Creating namespace: kuttl-test-vital-whale logger.go:42: 06:44:52 | 1-081_validate_applicationset_deployment/1- | starting test step 1- logger.go:42: 06:44:55 | 1-081_validate_applicationset_deployment/1- | test step completed 1- logger.go:42: 06:44:55 | 1-081_validate_applicationset_deployment | skipping kubernetes event logging logger.go:42: 06:44:55 | 1-081_validate_applicationset_deployment | Deleting namespace: kuttl-test-vital-whale === CONT kuttl/harness/1-063_validate_dex_liveness_probe logger.go:42: 06:45:01 | 1-063_validate_dex_liveness_probe | Creating namespace: kuttl-test-subtle-mastiff logger.go:42: 06:45:01 | 1-063_validate_dex_liveness_probe/1- | starting test step 1- I0220 06:45:02.169327 122 request.go:682] Waited for 1.032396827s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 06:45:06 | 1-104_validate_applicationset_tls_scm_volume_mount/99-delete | test step completed 99-delete logger.go:42: 06:45:06 | 1-104_validate_applicationset_tls_scm_volume_mount | skipping kubernetes event logging === CONT kuttl/harness/1-063_validate_dex_liveness_probe logger.go:42: 06:45:06 | 1-063_validate_dex_liveness_probe/1- | test step completed 1- logger.go:42: 06:45:06 | 1-063_validate_dex_liveness_probe | skipping kubernetes event logging logger.go:42: 06:45:06 | 1-063_validate_dex_liveness_probe | Deleting namespace: kuttl-test-subtle-mastiff === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 06:45:06 | 1-104_validate_applicationset_tls_scm_volume_mount | Deleting namespace: kuttl-test-evolving-goblin === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 06:45:11 | 1-080_validate_regex_support_argocd_rbac | Creating namespace: kuttl-test-vocal-bass logger.go:42: 06:45:11 | 1-080_validate_regex_support_argocd_rbac/1-install | starting test step 1-install === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance 2025/02/20 06:45:12 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/parallel/1-108_alternate_cluster_roles_cluster_scoped_instance/04-create-app.yaml I0220 06:45:12.976450 122 request.go:682] Waited for 1.032877763s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1?timeout=32s logger.go:42: 06:45:14 | 1-108_alternate_cluster_roles_cluster_scoped_instance | Creating namespace: kuttl-test-tough-sheepdog logger.go:42: 06:45:14 | 1-108_alternate_cluster_roles_cluster_scoped_instance/1-create-ns | starting test step 1-create-ns === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 06:45:14 | 1-080_validate_regex_support_argocd_rbac/1-install | ArgoCD:kuttl-test-vocal-bass/example-argocd created === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 06:45:17 | 1-108_alternate_cluster_roles_cluster_scoped_instance/1-create-ns | Namespace:/alternate-role created logger.go:42: 06:45:17 | 1-108_alternate_cluster_roles_cluster_scoped_instance/1-create-ns | test step completed 1-create-ns logger.go:42: 06:45:17 | 1-108_alternate_cluster_roles_cluster_scoped_instance/2-edit-subscription | starting test step 2-edit-subscription logger.go:42: 06:45:17 | 1-108_alternate_cluster_roles_cluster_scoped_instance/2-edit-subscription | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_CLUSTER_CONFIG_NAMESPACES", "value": "openshift-gitops, alternate-role"}]}}}' ] logger.go:42: 06:45:17 | 1-108_alternate_cluster_roles_cluster_scoped_instance/2-edit-subscription | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found case.go:364: failed in step 2-edit-subscription case.go:366: exit status 1 logger.go:42: 06:45:19 | 1-108_alternate_cluster_roles_cluster_scoped_instance | skipping kubernetes event logging logger.go:42: 06:45:19 | 1-108_alternate_cluster_roles_cluster_scoped_instance | Deleting namespace: kuttl-test-tough-sheepdog === CONT kuttl/harness/1-063_validate_statefulset_restart logger.go:42: 06:45:19 | 1-063_validate_statefulset_restart/1-install | test step completed 1-install logger.go:42: 06:45:19 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | starting test step 2-change_to_invalid_image logger.go:42: 06:45:19 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | running command: [sh -c set -e oc patch statefulset/example-argocd-application-controller \ -n $NAMESPACE \ --type "json" \ -p '[{"op":"replace","path":"/spec/template/spec/containers/0/image","value":"invalid_image"}]' sleep 10 ] logger.go:42: 06:45:20 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | statefulset.apps/example-argocd-application-controller patched === CONT kuttl/harness/1-079_validate_vars_for_notificaitons logger.go:42: 06:45:25 | 1-079_validate_vars_for_notificaitons | Creating namespace: kuttl-test-glorious-cardinal logger.go:42: 06:45:25 | 1-079_validate_vars_for_notificaitons/1-install | starting test step 1-install I0220 06:45:26.891492 122 request.go:682] Waited for 1.029860664s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 06:45:28 | 1-079_validate_vars_for_notificaitons/1-install | ArgoCD:kuttl-test-glorious-cardinal/example-argocd created === CONT kuttl/harness/1-063_validate_statefulset_restart logger.go:42: 06:45:32 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | test step completed 2-change_to_invalid_image logger.go:42: 06:45:32 | 1-063_validate_statefulset_restart/3-check_image_after_change | starting test step 3-check_image_after_change logger.go:42: 06:45:32 | 1-063_validate_statefulset_restart/3-check_image_after_change | running command: [sh -c set -e image=$(oc get statefulset/example-argocd-application-controller -o jsonpath='{.spec.template.spec.containers[].image}' -n $NAMESPACE) if test "${image}" == "invalid_image"; then echo "The application-controller statefulset is using 'invalid_image'" exit 1 fi exit 0 ] logger.go:42: 06:45:35 | 1-063_validate_statefulset_restart/3-check_image_after_change | test step completed 3-check_image_after_change logger.go:42: 06:45:35 | 1-063_validate_statefulset_restart | skipping kubernetes event logging logger.go:42: 06:45:35 | 1-063_validate_statefulset_restart | Deleting namespace: kuttl-test-amazed-ox === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 06:45:41 | 1-062_validate_extra_config | Creating namespace: kuttl-test-precious-bear logger.go:42: 06:45:42 | 1-062_validate_extra_config/1-argocd-with-extraconfig | starting test step 1-argocd-with-extraconfig I0220 06:45:43.061384 122 request.go:682] Waited for 1.032353824s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/rbac.authorization.k8s.io/v1?timeout=32s logger.go:42: 06:45:44 | 1-062_validate_extra_config/1-argocd-with-extraconfig | ArgoCD:kuttl-test-precious-bear/example-argocd created === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 06:45:46 | 1-080_validate_regex_support_argocd_rbac/1-install | test step completed 1-install logger.go:42: 06:45:46 | 1-080_validate_regex_support_argocd_rbac/2-modify-argocd | starting test step 2-modify-argocd logger.go:42: 06:45:49 | 1-080_validate_regex_support_argocd_rbac/2-modify-argocd | ArgoCD:kuttl-test-vocal-bass/example-argocd updated logger.go:42: 06:45:50 | 1-080_validate_regex_support_argocd_rbac/2-modify-argocd | test step completed 2-modify-argocd logger.go:42: 06:45:50 | 1-080_validate_regex_support_argocd_rbac/3-try-modify | starting test step 3-try-modify logger.go:42: 06:45:52 | 1-080_validate_regex_support_argocd_rbac/3-try-modify | ConfigMap:kuttl-test-vocal-bass/argocd-rbac-cm updated logger.go:42: 06:45:53 | 1-080_validate_regex_support_argocd_rbac/3-try-modify | test step completed 3-try-modify logger.go:42: 06:45:53 | 1-080_validate_regex_support_argocd_rbac/4-modify-argocd | starting test step 4-modify-argocd I0220 06:45:54.965583 122 request.go:682] Waited for 1.032332685s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s logger.go:42: 06:45:56 | 1-080_validate_regex_support_argocd_rbac/4-modify-argocd | ArgoCD:kuttl-test-vocal-bass/example-argocd updated logger.go:42: 06:45:57 | 1-080_validate_regex_support_argocd_rbac/4-modify-argocd | test step completed 4-modify-argocd logger.go:42: 06:45:57 | 1-080_validate_regex_support_argocd_rbac | skipping kubernetes event logging logger.go:42: 06:45:57 | 1-080_validate_regex_support_argocd_rbac | Deleting namespace: kuttl-test-vocal-bass === CONT kuttl/harness/1-079_validate_vars_for_notificaitons logger.go:42: 06:46:00 | 1-079_validate_vars_for_notificaitons/1-install | test step completed 1-install logger.go:42: 06:46:00 | 1-079_validate_vars_for_notificaitons/2-modify-argocd | starting test step 2-modify-argocd logger.go:42: 06:46:03 | 1-079_validate_vars_for_notificaitons/2-modify-argocd | ArgoCD:kuttl-test-glorious-cardinal/example-argocd updated === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 06:46:03 | 1-054_validate_deploymentconfig | Creating namespace: kuttl-test-master-beagle logger.go:42: 06:46:03 | 1-054_validate_deploymentconfig/1-install | starting test step 1-install I0220 06:46:04.981219 122 request.go:682] Waited for 1.281589807s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apps.openshift.io/v1?timeout=32s logger.go:42: 06:46:06 | 1-054_validate_deploymentconfig/1-install | ArgoCD:kuttl-test-master-beagle/argocd created === CONT kuttl/harness/1-079_validate_vars_for_notificaitons logger.go:42: 06:46:07 | 1-079_validate_vars_for_notificaitons/2-modify-argocd | test step completed 2-modify-argocd logger.go:42: 06:46:07 | 1-079_validate_vars_for_notificaitons | skipping kubernetes event logging logger.go:42: 06:46:07 | 1-079_validate_vars_for_notificaitons | Deleting namespace: kuttl-test-glorious-cardinal === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 06:46:13 | 1-061_validate_resource_tracking_method | Creating namespace: kuttl-test-fancy-bobcat logger.go:42: 06:46:13 | 1-061_validate_resource_tracking_method/1-install | starting test step 1-install I0220 06:46:15.028311 122 request.go:682] Waited for 1.379949405s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/cloud.network.openshift.io/v1?timeout=32s logger.go:42: 06:46:16 | 1-061_validate_resource_tracking_method/1-install | ArgoCD:kuttl-test-fancy-bobcat/argocd created === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 06:46:17 | 1-062_validate_extra_config/1-argocd-with-extraconfig | test step completed 1-argocd-with-extraconfig logger.go:42: 06:46:17 | 1-062_validate_extra_config/2-argocd-with-firstclass-and-extraconfig copy | starting test step 2-argocd-with-firstclass-and-extraconfig copy logger.go:42: 06:46:19 | 1-062_validate_extra_config/2-argocd-with-firstclass-and-extraconfig copy | ArgoCD:kuttl-test-precious-bear/example-argocd updated logger.go:42: 06:46:19 | 1-062_validate_extra_config/2-argocd-with-firstclass-and-extraconfig copy | test step completed 2-argocd-with-firstclass-and-extraconfig copy logger.go:42: 06:46:19 | 1-062_validate_extra_config/3-update-configmap | starting test step 3-update-configmap logger.go:42: 06:46:22 | 1-062_validate_extra_config/3-update-configmap | ConfigMap:kuttl-test-precious-bear/argocd-cm updated logger.go:42: 06:46:23 | 1-062_validate_extra_config/3-update-configmap | test step completed 3-update-configmap logger.go:42: 06:46:23 | 1-062_validate_extra_config/4-create-dex-without-extraconfig | starting test step 4-create-dex-without-extraconfig I0220 06:46:25.034137 122 request.go:682] Waited for 1.629031771s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apiextensions.k8s.io/v1?timeout=32s logger.go:42: 06:46:25 | 1-062_validate_extra_config/4-create-dex-without-extraconfig | ArgoCD:kuttl-test-precious-bear/example-argocd updated logger.go:42: 06:46:26 | 1-062_validate_extra_config/4-create-dex-without-extraconfig | test step completed 4-create-dex-without-extraconfig logger.go:42: 06:46:26 | 1-062_validate_extra_config/5-override-dex-using-extraconfig | starting test step 5-override-dex-using-extraconfig === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 06:46:29 | 1-054_validate_deploymentconfig/1-install | test step completed 1-install logger.go:42: 06:46:29 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | starting test step 2-install-deploymentconfig logger.go:42: 06:46:29 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | running command: [sh -c set -eo pipefail # Install ArgoCD Application with 2 replicas cat << EOF | oc apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: app-deploymentconfig namespace: ${NAMESPACE} spec: project: default source: repoURL: https://github.com/redhat-developer/gitops-operator path: test/examples/deploymentconfig-example targetRevision: "HEAD" destination: server: https://kubernetes.default.svc namespace: ${NAMESPACE} syncPolicy: automated: {} EOF # Give some time to Application before asserting sleep 10 exit 0 ] === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 06:46:29 | 1-062_validate_extra_config/5-override-dex-using-extraconfig | ArgoCD:kuttl-test-precious-bear/example-argocd updated logger.go:42: 06:46:29 | 1-062_validate_extra_config/5-override-dex-using-extraconfig | test step completed 5-override-dex-using-extraconfig logger.go:42: 06:46:29 | 1-062_validate_extra_config | skipping kubernetes event logging logger.go:42: 06:46:29 | 1-062_validate_extra_config | Deleting namespace: kuttl-test-precious-bear === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 06:46:29 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | application.argoproj.io/app-deploymentconfig created I0220 06:46:40.942134 122 request.go:682] Waited for 1.033231598s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/admissionregistration.k8s.io/v1?timeout=32s logger.go:42: 06:46:42 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | test step completed 2-install-deploymentconfig logger.go:42: 06:46:42 | 1-054_validate_deploymentconfig/3- | starting test step 3- logger.go:42: 06:46:47 | 1-054_validate_deploymentconfig/3- | test step completed 3- logger.go:42: 06:46:47 | 1-054_validate_deploymentconfig/4-scaledown-deploymentconfig | starting test step 4-scaledown-deploymentconfig === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 06:46:49 | 1-061_validate_resource_tracking_method/1-install | test step completed 1-install logger.go:42: 06:46:49 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | starting test step 2-patch_cm_annotation logger.go:42: 06:46:49 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"resourceTrackingMethod":"annotation"}}' -n $NAMESPACE ] === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 06:46:49 | 1-054_validate_deploymentconfig/4-scaledown-deploymentconfig | Application:kuttl-test-master-beagle/app-deploymentconfig updated === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 06:46:50 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | argocd.argoproj.io/argocd patched I0220 06:46:51.347325 122 request.go:682] Waited for 1.03283313s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 06:46:52 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | test step completed 2-patch_cm_annotation logger.go:42: 06:46:52 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | starting test step 3-patch_cm_annotation_label logger.go:42: 06:46:52 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"resourceTrackingMethod":"annotation+label"}}' -n $NAMESPACE ] === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 06:46:52 | 1-054_validate_deploymentconfig/4-scaledown-deploymentconfig | test step completed 4-scaledown-deploymentconfig logger.go:42: 06:46:52 | 1-054_validate_deploymentconfig | skipping kubernetes event logging logger.go:42: 06:46:52 | 1-054_validate_deploymentconfig | Deleting namespace: kuttl-test-master-beagle === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 06:46:53 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | argocd.argoproj.io/argocd patched logger.go:42: 06:46:55 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | test step completed 3-patch_cm_annotation_label logger.go:42: 06:46:55 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | starting test step 4-patch_cm_invalid_method logger.go:42: 06:46:55 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"resourceTrackingMethod":"invalid_method"}}' -n $NAMESPACE ] logger.go:42: 06:46:56 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | argocd.argoproj.io/argocd patched logger.go:42: 06:46:59 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | test step completed 4-patch_cm_invalid_method logger.go:42: 06:46:59 | 1-061_validate_resource_tracking_method | skipping kubernetes event logging logger.go:42: 06:46:59 | 1-061_validate_resource_tracking_method | Deleting namespace: kuttl-test-fancy-bobcat === CONT kuttl/harness/1-053_validate_cluster_admin_rbac logger.go:42: 06:47:05 | 1-053_validate_cluster_admin_rbac | Creating namespace: kuttl-test-star-colt logger.go:42: 06:47:05 | 1-053_validate_cluster_admin_rbac/1- | starting test step 1- I0220 06:47:06.267656 122 request.go:682] Waited for 1.032435637s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/admissionregistration.k8s.io/v1?timeout=32s logger.go:42: 06:47:07 | 1-053_validate_cluster_admin_rbac/1- | test step completed 1- logger.go:42: 06:47:07 | 1-053_validate_cluster_admin_rbac/2- | starting test step 2- logger.go:42: 06:47:10 | 1-053_validate_cluster_admin_rbac/2- | test step completed 2- logger.go:42: 06:47:10 | 1-053_validate_cluster_admin_rbac | skipping kubernetes event logging logger.go:42: 06:47:10 | 1-053_validate_cluster_admin_rbac | Deleting namespace: kuttl-test-star-colt === CONT kuttl/harness/1-058_validate_prometheus_rule logger.go:42: 06:47:16 | 1-058_validate_prometheus_rule | Creating namespace: kuttl-test-living-basilisk logger.go:42: 06:47:16 | 1-058_validate_prometheus_rule/1-install | starting test step 1-install I0220 06:47:17.181997 122 request.go:682] Waited for 1.031029357s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/rbac.authorization.k8s.io/v1?timeout=32s logger.go:42: 06:47:18 | 1-058_validate_prometheus_rule/1-install | ArgoCD:kuttl-test-living-basilisk/example-argocd created === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 06:47:38 | 1-075_validate_dex_anyuid | Creating namespace: kuttl-test-intimate-grouper logger.go:42: 06:47:38 | 1-075_validate_dex_anyuid/1-install | starting test step 1-install I0220 06:47:39.462858 122 request.go:682] Waited for 1.032785078s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/project.openshift.io/v1?timeout=32s logger.go:42: 06:47:40 | 1-075_validate_dex_anyuid/1-install | ArgoCD:kuttl-test-intimate-grouper/argocd created === CONT kuttl/harness/1-058_validate_prometheus_rule logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/1-install | test step completed 1-install logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | starting test step 2-wait logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running $NAMESPACE ] logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | Waiting until all pods in namespace kuttl-test-living-basilisk are up logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | All pods are up: logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | example-argocd-application-controller-0 1/1 Running 0 32s logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | example-argocd-redis-6dc7b59d8f-p8qjq 1/1 Running 0 32s logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | example-argocd-repo-server-7d7b8cfbd-fqqkq 1/1 Running 0 32s logger.go:42: 06:47:51 | 1-058_validate_prometheus_rule/2-wait | example-argocd-server-8d94cc544-k9lcc 1/1 Running 0 32s I0220 06:47:52.606098 122 request.go:682] Waited for 1.031444899s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1?timeout=32s logger.go:42: 06:47:54 | 1-058_validate_prometheus_rule/2-wait | test step completed 2-wait logger.go:42: 06:47:54 | 1-058_validate_prometheus_rule/3-create_application | starting test step 3-create_application logger.go:42: 06:47:54 | 1-058_validate_prometheus_rule/3-create_application | running command: [sh -c set -e cat << EOF | oc apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: test-1-58-custom namespace: ${NAMESPACE} spec: project: default source: repoURL: https://github.com/jaideepr97/gitops-examples path: bgd-k8s targetRevision: "HEAD" destination: server: "https://kubernetes.default.svc" namespace: ${NAMESPACE} syncPolicy: automated: {} EOF ] logger.go:42: 06:47:54 | 1-058_validate_prometheus_rule/3-create_application | application.argoproj.io/test-1-58-custom created logger.go:42: 06:47:57 | 1-058_validate_prometheus_rule/3-create_application | test step completed 3-create_application logger.go:42: 06:47:57 | 1-058_validate_prometheus_rule | skipping kubernetes event logging logger.go:42: 06:47:57 | 1-058_validate_prometheus_rule | Deleting namespace: kuttl-test-living-basilisk === CONT kuttl/harness/1-077_validate_disable_dex_removed logger.go:42: 06:48:03 | 1-077_validate_disable_dex_removed | Creating namespace: kuttl-test-warm-griffon logger.go:42: 06:48:03 | 1-077_validate_disable_dex_removed/1-check_disable_dex_removed | starting test step 1-check_disable_dex_removed logger.go:42: 06:48:03 | 1-077_validate_disable_dex_removed/1-check_disable_dex_removed | running command: [sh -c set -e operator_name_version=$(oc get subscription/openshift-gitops-operator -n openshift-operators -o jsonpath='{.status.installedCSV}') if [ "$(oc get csv/"${operator_name_version}" -n openshift-operators -o jsonpath='{.spec.install.spec.deployments[].spec.template.spec.containers[].env}' | grep DISABLE_DEX)" != '' ]; then echo "DISABLE_DEX env var is still present in the operator CSV." exit 1 fi ] logger.go:42: 06:48:03 | 1-077_validate_disable_dex_removed/1-check_disable_dex_removed | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found I0220 06:48:04.516741 122 request.go:682] Waited for 1.032598855s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/storage.k8s.io/v1beta1?timeout=32s case.go:364: failed in step 1-check_disable_dex_removed case.go:366: exit status 1 logger.go:42: 06:48:05 | 1-077_validate_disable_dex_removed | skipping kubernetes event logging logger.go:42: 06:48:05 | 1-077_validate_disable_dex_removed | Deleting namespace: kuttl-test-warm-griffon === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:48:11 | 1-057_validate_notifications | Creating namespace: kuttl-test-direct-beetle logger.go:42: 06:48:11 | 1-057_validate_notifications/1-install | starting test step 1-install === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 06:48:14 | 1-075_validate_dex_anyuid/1-install | test step completed 1-install logger.go:42: 06:48:14 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | starting test step 2-change_dex_scc_to_anyuid logger.go:42: 06:48:14 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | running command: [sh -c set -e oc adm policy add-scc-to-user anyuid -z argocd-argocd-dex-server -n $NAMESPACE ] === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:48:14 | 1-057_validate_notifications/1-install | ArgoCD:kuttl-test-direct-beetle/argocd created logger.go:42: 06:48:14 | 1-057_validate_notifications/1-install | Service:kuttl-test-direct-beetle/smtp4dev created logger.go:42: 06:48:14 | 1-057_validate_notifications/1-install | Deployment:kuttl-test-direct-beetle/smtp4dev created === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 06:48:14 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "argocd-argocd-dex-server" I0220 06:48:15.788240 122 request.go:682] Waited for 1.033584981s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1beta1?timeout=32s logger.go:42: 06:48:17 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | test step completed 2-change_dex_scc_to_anyuid logger.go:42: 06:48:17 | 1-075_validate_dex_anyuid/3-restart_dex_pod | starting test step 3-restart_dex_pod logger.go:42: 06:48:17 | 1-075_validate_dex_anyuid/3-restart_dex_pod | running command: [sh -c set -e oc rollout restart deployment/argocd-dex-server -n $NAMESPACE ] logger.go:42: 06:48:17 | 1-075_validate_dex_anyuid/3-restart_dex_pod | deployment.apps/argocd-dex-server restarted logger.go:42: 06:48:22 | 1-075_validate_dex_anyuid/3-restart_dex_pod | test step completed 3-restart_dex_pod logger.go:42: 06:48:22 | 1-075_validate_dex_anyuid | skipping kubernetes event logging logger.go:42: 06:48:22 | 1-075_validate_dex_anyuid | Deleting namespace: kuttl-test-intimate-grouper === CONT kuttl/harness/1-052_validate_rolebinding_number logger.go:42: 06:48:28 | 1-052_validate_rolebinding_number | Creating namespace: kuttl-test-humane-troll logger.go:42: 06:48:28 | 1-052_validate_rolebinding_number/1-label_namespace | starting test step 1-label_namespace logger.go:42: 06:48:28 | 1-052_validate_rolebinding_number/1-label_namespace | running command: [sh -c oc label namespace $NAMESPACE argocd.argoproj.io/managed-by=openshift-gitops ] logger.go:42: 06:48:28 | 1-052_validate_rolebinding_number/1-label_namespace | namespace/kuttl-test-humane-troll labeled I0220 06:48:29.982416 122 request.go:682] Waited for 1.03305678s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/user.openshift.io/v1?timeout=32s logger.go:42: 06:48:31 | 1-052_validate_rolebinding_number/1-label_namespace | test step completed 1-label_namespace logger.go:42: 06:48:31 | 1-052_validate_rolebinding_number/2-check_rolebindings | starting test step 2-check_rolebindings logger.go:42: 06:48:31 | 1-052_validate_rolebinding_number/2-check_rolebindings | running command: [sh -c set -eo pipefail # Expected and Current RoleBindings expected_rb=( "openshift-gitops-argocd-application-controller" "openshift-gitops-argocd-server" ) # Check that eventually the current role bindings equal the expected role # bindings. Timeout after 60 seconds timer=0 current_rb=( $(oc get rolebindings -n "${NAMESPACE}" | awk '/gitops/ {print $1}' | sort) ) while [[ "${current_rb[@]}" != "${expected_rb[@]}" ]]; do if [[ $timer -eq 60 ]]; then echo "timed out waiting for current role bindings to equal expected role bindings" echo "current role bindings: ${current_rb[*]}" echo "expected role bindings: ${expected_rb[*]}" exit 1 fi timer=$((timer+5)) sleep 5 current_rb=( $(oc get rolebindings -n "${NAMESPACE}" | awk '/gitops/ {print $1}' | sort) ) done # Check that the expected role bindings continue to exist for at least the # next 20 seconds timer=0 while [ true ]; do if [[ $timer -eq 20 ]]; then break fi current_rb=( $(oc get rolebindings -n "${NAMESPACE}" | awk '/gitops/ {print $1}' | sort) ) if [[ "${current_rb[@]}" != "${expected_rb[@]}" ]]; then echo "current role bindings have changed, now not equal to expected role bindings" echo "current role bindings: ${current_rb[*]}" echo "expected role bindings: ${expected_rb[*]}" exit 1 fi timer=$((timer+5)) sleep 5 done ] === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:48:45 | 1-057_validate_notifications/1-install | test step completed 1-install logger.go:42: 06:48:45 | 1-057_validate_notifications/2-update-notifications-cm | starting test step 2-update-notifications-cm logger.go:42: 06:48:45 | 1-057_validate_notifications/2-update-notifications-cm | running command: [sh -c sleep 15] === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 06:48:45 | 1-074_validate_terminating_namespace_block | Creating namespace: kuttl-test-sharing-ladybird logger.go:42: 06:48:45 | 1-074_validate_terminating_namespace_block/1-install | starting test step 1-install I0220 06:48:47.005694 122 request.go:682] Waited for 1.029336998s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1?timeout=32s logger.go:42: 06:48:48 | 1-074_validate_terminating_namespace_block/1-install | Namespace:/gitops-2242-ns-main created logger.go:42: 06:48:48 | 1-074_validate_terminating_namespace_block/1-install | ArgoCD:gitops-2242-ns-main/gitops-2242-argocd created logger.go:42: 06:48:48 | 1-074_validate_terminating_namespace_block/1-install | RoleBinding:gitops-2242-ns-main/grant-argocd created logger.go:42: 06:48:48 | 1-074_validate_terminating_namespace_block/1-install | test step completed 1-install logger.go:42: 06:48:48 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | starting test step 2-managed-ns-cm logger.go:42: 06:48:51 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | Namespace:/gitops-2242-ns-first created logger.go:42: 06:48:51 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | ConfigMap:gitops-2242-ns-first/my-config-map-2 created logger.go:42: 06:48:51 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | test step completed 2-managed-ns-cm logger.go:42: 06:48:51 | 1-074_validate_terminating_namespace_block/3-delete-ns | starting test step 3-delete-ns logger.go:42: 06:48:51 | 1-074_validate_terminating_namespace_block/3-delete-ns | running command: [sh -c set -e oc delete ns gitops-2242-ns-first --wait=false exit 0 ] logger.go:42: 06:48:51 | 1-074_validate_terminating_namespace_block/3-delete-ns | namespace "gitops-2242-ns-first" deleted logger.go:42: 06:48:54 | 1-074_validate_terminating_namespace_block/3-delete-ns | test step completed 3-delete-ns logger.go:42: 06:48:54 | 1-074_validate_terminating_namespace_block/4-create-ns | starting test step 4-create-ns I0220 06:48:57.863166 122 request.go:682] Waited for 1.041396507s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-052_validate_rolebinding_number logger.go:42: 06:48:59 | 1-052_validate_rolebinding_number/2-check_rolebindings | test step completed 2-check_rolebindings logger.go:42: 06:48:59 | 1-052_validate_rolebinding_number | skipping kubernetes event logging logger.go:42: 06:48:59 | 1-052_validate_rolebinding_number | Deleting namespace: kuttl-test-humane-troll === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 06:48:59 | 1-074_validate_terminating_namespace_block/4-create-ns | Namespace:/gitops-2242-ns-second created logger.go:42: 06:49:00 | 1-074_validate_terminating_namespace_block/4-create-ns | test step completed 4-create-ns logger.go:42: 06:49:00 | 1-074_validate_terminating_namespace_block/99-cleanup | starting test step 99-cleanup logger.go:42: 06:49:00 | 1-074_validate_terminating_namespace_block/99-cleanup | running command: [sh -c set -e # Removing finalizer from cm oc patch cm/my-config-map-2 -n gitops-2242-ns-first --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' # Deleting namespaces oc delete ns gitops-2242-ns-first --wait=false oc delete ns gitops-2242-ns-second --wait=false oc delete ns gitops-2242-ns-main --wait=false exit 0 ] === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:49:00 | 1-057_validate_notifications/2-update-notifications-cm | running command: [sh -c set -e kubectl patch notificationsconfiguration default-notifications-configuration -n $NAMESPACE --type='json' -p='[{"op": "add", "path": "/spec/services", "value": {"service.email.gmail": "{host: smtp4dev, port: 2525, from: fake@email.com }" } }]' ] === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 06:49:00 | 1-074_validate_terminating_namespace_block/99-cleanup | configmap/my-config-map-2 patched logger.go:42: 06:49:01 | 1-074_validate_terminating_namespace_block/99-cleanup | namespace "gitops-2242-ns-first" deleted === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:49:01 | 1-057_validate_notifications/2-update-notifications-cm | notificationsconfiguration.argoproj.io/default-notifications-configuration patched logger.go:42: 06:49:01 | 1-057_validate_notifications/2-update-notifications-cm | running command: [sh -c sleep 5] === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 06:49:02 | 1-074_validate_terminating_namespace_block/99-cleanup | namespace "gitops-2242-ns-second" deleted logger.go:42: 06:49:02 | 1-074_validate_terminating_namespace_block/99-cleanup | namespace "gitops-2242-ns-main" deleted logger.go:42: 06:49:05 | 1-074_validate_terminating_namespace_block/99-cleanup | test step completed 99-cleanup logger.go:42: 06:49:05 | 1-074_validate_terminating_namespace_block | skipping kubernetes event logging logger.go:42: 06:49:05 | 1-074_validate_terminating_namespace_block | Deleting namespace: kuttl-test-sharing-ladybird === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 06:49:05 | 1-055_validate_notification_controller | Creating namespace: kuttl-test-sweet-redfish logger.go:42: 06:49:05 | 1-055_validate_notification_controller/1-install | starting test step 1-install logger.go:42: 06:49:07 | 1-055_validate_notification_controller/1-install | ArgoCD:kuttl-test-sweet-redfish/example-argocd created I0220 06:49:08.823168 122 request.go:682] Waited for 1.033260805s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/discovery.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:49:10 | 1-057_validate_notifications/2-update-notifications-cm | test step completed 2-update-notifications-cm logger.go:42: 06:49:10 | 1-057_validate_notifications/3-create-app | starting test step 3-create-app logger.go:42: 06:49:10 | 1-057_validate_notifications/3-create-app | running command: [sh -c set -e cat << EOF | oc apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app-3 namespace: $NAMESPACE annotations: "notifications.argoproj.io/subscribe.on-created.gmail": "jdfake@email.com" spec: destination: namespace: $NAMESPACE server: https://kubernetes.default.svc project: default source: repoURL: https://github.com/redhat-developer/gitops-operator path: test/examples/nginx targetRevision: HEAD EOF ] logger.go:42: 06:49:10 | 1-057_validate_notifications/3-create-app | application.argoproj.io/my-app-3 created logger.go:42: 06:49:10 | 1-057_validate_notifications/3-create-app | running command: [sh -c sleep 5] === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 06:49:10 | 1-073_validate_rhsso | Creating namespace: kuttl-test-helped-peacock logger.go:42: 06:49:11 | 1-073_validate_rhsso/1-argocd-rhsso | starting test step 1-argocd-rhsso logger.go:42: 06:49:13 | 1-073_validate_rhsso/1-argocd-rhsso | ArgoCD:kuttl-test-helped-peacock/example-argocd-keycloak created === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:49:18 | 1-057_validate_notifications/3-create-app | test step completed 3-create-app logger.go:42: 06:49:18 | 1-057_validate_notifications/4-delete-app | starting test step 4-delete-app logger.go:42: 06:49:18 | 1-057_validate_notifications/4-delete-app | running command: [sh -c set -e kubectl delete -n $NAMESPACE application.argoproj.io my-app-3 ] logger.go:42: 06:49:18 | 1-057_validate_notifications/4-delete-app | application.argoproj.io "my-app-3" deleted logger.go:42: 06:49:18 | 1-057_validate_notifications/4-delete-app | running command: [sh -c sleep 5] === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 06:49:19 | 1-055_validate_notification_controller/1-install | test step completed 1-install logger.go:42: 06:49:19 | 1-055_validate_notification_controller/2-enable_notification | starting test step 2-enable_notification I0220 06:49:20.257787 122 request.go:682] Waited for 1.032539911s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 06:49:21 | 1-055_validate_notification_controller/2-enable_notification | ArgoCD:kuttl-test-sweet-redfish/example-argocd updated logger.go:42: 06:49:26 | 1-055_validate_notification_controller/2-enable_notification | test step completed 2-enable_notification logger.go:42: 06:49:26 | 1-055_validate_notification_controller/3-add-sleep | starting test step 3-add-sleep logger.go:42: 06:49:26 | 1-055_validate_notification_controller/3-add-sleep | running command: [sh -c sleep 20] === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 06:49:26 | 1-057_validate_notifications/4-delete-app | test step completed 4-delete-app logger.go:42: 06:49:26 | 1-057_validate_notifications/5-verify-email | starting test step 5-verify-email logger.go:42: 06:49:26 | 1-057_validate_notifications/5-verify-email | running command: [sh -c set -e smtp4dev_pod=$(kubectl get pod -l=app=smtp4dev -o NAME -n $NAMESPACE) exit_code=$(kubectl -n $NAMESPACE exec -i --stdin "${smtp4dev_pod}" -- /bin/bash \ -c 'if [[ $(grep -rnw /tmp -e "Subject: Application my-app-3 has been created.") ]]; then exit 0; else exit 1; fi') if [ $exit_code=0 ]; then exit 0 else exit 1 fi ] logger.go:42: 06:49:29 | 1-057_validate_notifications/5-verify-email | test step completed 5-verify-email logger.go:42: 06:49:29 | 1-057_validate_notifications | skipping kubernetes event logging logger.go:42: 06:49:29 | 1-057_validate_notifications | Deleting namespace: kuttl-test-direct-beetle === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:49:34 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | running command: [sh -c set -eo pipefail cleanup() { rm alerts.json } trap cleanup INT TERM EXIT oc whoami jq '.data.alerts' <<< "$(oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -s 'http://localhost:9090/api/v1/alerts')" > alerts.json result=$(jq --arg NAMESPACE "$NAMESPACE" ' .[] | select( .labels.deployment == "argocd-applicationset-controller" and .labels.namespace == $NAMESPACE and .labels.alertname == "ApplicationSetControllerNotReady" and .state == "firing")' alerts.json) [[ -n "$result" ]] && exit 0 || exit 1 ] logger.go:42: 06:49:34 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | kube:admin === CONT kuttl/harness/1-055_drop_resource_customizations I0220 06:49:36.650672 122 request.go:682] Waited for 1.032315979s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha2?timeout=32s === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:49:38 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | test step completed 2-verify-alert logger.go:42: 06:49:38 | 1-092_validate_workload_status_monitoring_alert/3-disable-monitoring | starting test step 3-disable-monitoring === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 06:49:40 | 1-055_drop_resource_customizations | Creating namespace: kuttl-test-arriving-boar logger.go:42: 06:49:40 | 1-055_drop_resource_customizations/1-alpha-argocd | starting test step 1-alpha-argocd === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:49:40 | 1-092_validate_workload_status_monitoring_alert/3-disable-monitoring | ArgoCD:kuttl-test-giving-seasnail/argocd updated logger.go:42: 06:49:43 | 1-092_validate_workload_status_monitoring_alert/3-disable-monitoring | test step completed 3-disable-monitoring logger.go:42: 06:49:43 | 1-092_validate_workload_status_monitoring_alert | skipping kubernetes event logging === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 06:49:43 | 1-055_drop_resource_customizations/1-alpha-argocd | ArgoCD:kuttl-test-arriving-boar/argocd created === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 06:49:43 | 1-092_validate_workload_status_monitoring_alert | Deleting namespace: kuttl-test-giving-seasnail I0220 06:49:47.459666 122 request.go:682] Waited for 1.032487672s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 06:49:48 | 1-055_validate_notification_controller/3-add-sleep | ArgoCD:kuttl-test-sweet-redfish/example-argocd updated logger.go:42: 06:49:49 | 1-055_validate_notification_controller/3-add-sleep | test step completed 3-add-sleep logger.go:42: 06:49:49 | 1-055_validate_notification_controller | skipping kubernetes event logging logger.go:42: 06:49:49 | 1-055_validate_notification_controller | Deleting namespace: kuttl-test-sweet-redfish === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 06:49:49 | 1-072_validate_liveness_probe_removed | Creating namespace: kuttl-test-romantic-quail logger.go:42: 06:49:49 | 1-072_validate_liveness_probe_removed/1-install | starting test step 1-install logger.go:42: 06:49:51 | 1-072_validate_liveness_probe_removed/1-install | ArgoCD:kuttl-test-romantic-quail/argocd created === CONT kuttl/harness/1-051-validate_csv_permissions logger.go:42: 06:49:55 | 1-051-validate_csv_permissions | Creating namespace: kuttl-test-holy-moray logger.go:42: 06:49:55 | 1-051-validate_csv_permissions/1-validate | starting test step 1-validate logger.go:42: 06:49:55 | 1-051-validate_csv_permissions/1-validate | running command: [sh -c set -e res=$(oc auth can-i delete resourcequotas -n openshift-gitops --as system:serviceaccount:openshift-operators:openshift-gitops-operator-controller-manager) if test "$res" != "yes"; then echo "Can't delete resourcequotas" exit 1 fi ] I0220 06:49:57.498031 122 request.go:682] Waited for 1.832192736s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/config.openshift.io/v1?timeout=32s case.go:364: failed in step 1-validate case.go:366: exit status 1 logger.go:42: 06:49:58 | 1-051-validate_csv_permissions | skipping kubernetes event logging logger.go:42: 06:49:58 | 1-051-validate_csv_permissions | Deleting namespace: kuttl-test-holy-moray === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 06:50:03 | 1-072_validate_liveness_probe_removed/1-install | test step completed 1-install logger.go:42: 06:50:03 | 1-072_validate_liveness_probe_removed/2-check_liveness_probe_removed | starting test step 2-check_liveness_probe_removed logger.go:42: 06:50:03 | 1-072_validate_liveness_probe_removed/2-check_liveness_probe_removed | running command: [sh -c set -e if ! test -z $(oc get statefulset.apps/argocd-application-controller -n $NAMESPACE -o jsonpath='{.spec.template.spec.containers[].livenessProbe}'); then echo "livenessProbe should not exist on statefulset.apps/argocd-application-controller." exit 1 fi ] === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 06:50:06 | 1-072_validate_liveness_probe_removed/2-check_liveness_probe_removed | test step completed 2-check_liveness_probe_removed logger.go:42: 06:50:06 | 1-072_validate_liveness_probe_removed | skipping kubernetes event logging === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 06:50:06 | 1-068_validate_redis_secure_comm_autotls_no_ha | Creating namespace: kuttl-test-pumped-gecko logger.go:42: 06:50:06 | 1-068_validate_redis_secure_comm_autotls_no_ha/1-install | starting test step 1-install === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 06:50:06 | 1-072_validate_liveness_probe_removed | Deleting namespace: kuttl-test-romantic-quail I0220 06:50:07.540628 122 request.go:682] Waited for 1.232805338s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1alpha5?timeout=32s === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 06:50:08 | 1-068_validate_redis_secure_comm_autotls_no_ha/1-install | ArgoCD:kuttl-test-pumped-gecko/argocd created === CONT kuttl/harness/1-050_validate_sso logger.go:42: 06:50:12 | 1-050_validate_sso | Creating namespace: kuttl-test-main-squid logger.go:42: 06:50:12 | 1-050_validate_sso/1-install | starting test step 1-install === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 06:50:15 | 1-055_drop_resource_customizations/1-alpha-argocd | test step completed 1-alpha-argocd logger.go:42: 06:50:15 | 1-055_drop_resource_customizations/2- | starting test step 2- === CONT kuttl/harness/1-050_validate_sso logger.go:42: 06:50:15 | 1-050_validate_sso/1-install | ArgoCD:kuttl-test-main-squid/argocd created I0220 06:50:17.544170 122 request.go:682] Waited for 2.232803359s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/migration.k8s.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 06:50:17 | 1-055_drop_resource_customizations/2- | test step completed 2- logger.go:42: 06:50:17 | 1-055_drop_resource_customizations | skipping kubernetes event logging logger.go:42: 06:50:17 | 1-055_drop_resource_customizations | Deleting namespace: kuttl-test-arriving-boar === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 06:50:23 | 1-069_validate_redis_secure_comm_autotls_ha | Creating namespace: kuttl-test-major-blowfish logger.go:42: 06:50:23 | 1-069_validate_redis_secure_comm_autotls_ha/1-install | starting test step 1-install logger.go:42: 06:50:26 | 1-069_validate_redis_secure_comm_autotls_ha/1-install | ArgoCD:kuttl-test-major-blowfish/argocd created === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 06:50:40 | 1-068_validate_redis_secure_comm_autotls_no_ha/1-install | test step completed 1-install logger.go:42: 06:50:40 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | starting test step 2-enable_autotls logger.go:42: 06:50:40 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"redis":{"autotls":"openshift"}}}' -n $NAMESPACE ] logger.go:42: 06:50:41 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | argocd.argoproj.io/argocd patched I0220 06:50:42.234784 122 request.go:682] Waited for 1.029686517s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/scheduling.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-050_validate_sso logger.go:42: 06:50:46 | 1-050_validate_sso/1-install | test step completed 1-install logger.go:42: 06:50:46 | 1-050_validate_sso/2-remove-spec-sso | starting test step 2-remove-spec-sso logger.go:42: 06:50:46 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso"}]' ] logger.go:42: 06:50:47 | 1-050_validate_sso/2-remove-spec-sso | argocd.argoproj.io/argocd patched logger.go:42: 06:50:47 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c sleep 10] === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 06:50:54 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | test step completed 2-enable_autotls logger.go:42: 06:50:54 | 1-068_validate_redis_secure_comm_autotls_no_ha/3-check_secret | starting test step 3-check_secret logger.go:42: 06:50:54 | 1-068_validate_redis_secure_comm_autotls_no_ha/3-check_secret | running command: [sh -c set -e secret_type="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{.type}}')" secret_len="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{len .data}}')" expected_secret_type="kubernetes.io/tls" expected_secret_len=2 if test ${secret_type} != ${expected_secret_type}; then echo "argocd-operator-redis-tls secret type is ${secret_type} and should be ${expected_secret_type}" exit 1 fi if test ${secret_len} != ${expected_secret_len}; then echo "argocd-operator-redis-tls secret length is ${secret_len} and should be ${expected_secret_len}" exit 1 fi ] I0220 06:50:56.464084 122 request.go:682] Waited for 1.000037707s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/argoproj.io/v1alpha1?timeout=32s logger.go:42: 06:50:57 | 1-068_validate_redis_secure_comm_autotls_no_ha/3-check_secret | test step completed 3-check_secret logger.go:42: 06:50:57 | 1-068_validate_redis_secure_comm_autotls_no_ha/4- | starting test step 4- logger.go:42: 06:51:02 | 1-068_validate_redis_secure_comm_autotls_no_ha/4- | test step completed 4- logger.go:42: 06:51:02 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | starting test step 5-check_crt_files logger.go:42: 06:51:02 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | running command: [sh -c set -e oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-server -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/server/tls/redis/tls.crt oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-repo-server -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/reposerver/tls/redis/tls.crt oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-redis -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/redis/tls/tls.crt oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-application-controller -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/controller/tls/redis/tls.crt ] === CONT kuttl/harness/1-050_validate_sso logger.go:42: 06:51:03 | 1-050_validate_sso/2-remove-spec-sso | test step completed 2-remove-spec-sso logger.go:42: 06:51:03 | 1-050_validate_sso/7-provider-keycloak-dex-spec | starting test step 7-provider-keycloak-dex-spec === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 06:51:04 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/server/tls/redis/tls.crt logger.go:42: 06:51:04 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | Defaulted container "argocd-repo-server" out of: argocd-repo-server, copyutil (init) logger.go:42: 06:51:05 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/reposerver/tls/redis/tls.crt === CONT kuttl/harness/1-050_validate_sso logger.go:42: 06:51:05 | 1-050_validate_sso/7-provider-keycloak-dex-spec | ArgoCD:kuttl-test-main-squid/argocd updated logger.go:42: 06:51:05 | 1-050_validate_sso/7-provider-keycloak-dex-spec | test step completed 7-provider-keycloak-dex-spec logger.go:42: 06:51:05 | 1-050_validate_sso/8-remove-spec-dex | starting test step 8-remove-spec-dex logger.go:42: 06:51:05 | 1-050_validate_sso/8-remove-spec-dex | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso/dex"}]' ] logger.go:42: 06:51:06 | 1-050_validate_sso/8-remove-spec-dex | argocd.argoproj.io/argocd patched === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 06:51:06 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/redis/tls/tls.crt I0220 06:51:07.488557 122 request.go:682] Waited for 1.027745725s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/coordination.k8s.io/v1?timeout=32s logger.go:42: 06:51:07 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/controller/tls/redis/tls.crt logger.go:42: 06:51:11 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | test step completed 5-check_crt_files logger.go:42: 06:51:11 | 1-068_validate_redis_secure_comm_autotls_no_ha | skipping kubernetes event logging logger.go:42: 06:51:11 | 1-068_validate_redis_secure_comm_autotls_no_ha | Deleting namespace: kuttl-test-pumped-gecko === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 06:51:17 | 1-066_validate_redis_secure_comm_no_autotls_no_ha | Creating namespace: kuttl-test-romantic-phoenix logger.go:42: 06:51:17 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/1-install | starting test step 1-install I0220 06:51:18.891285 122 request.go:682] Waited for 1.032094402s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/storage.k8s.io/v1?timeout=32s logger.go:42: 06:51:20 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/1-install | ArgoCD:kuttl-test-romantic-phoenix/argocd created === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 06:51:29 | 1-073_validate_rhsso/1-argocd-rhsso | test step completed 1-argocd-rhsso logger.go:42: 06:51:29 | 1-073_validate_rhsso/2-sleep | starting test step 2-sleep logger.go:42: 06:51:29 | 1-073_validate_rhsso/2-sleep | running command: [sh -c sleep 60] === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 06:51:51 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/1-install | test step completed 1-install logger.go:42: 06:51:51 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | starting test step 2-generate_cert logger.go:42: 06:51:51 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | running command: [sh -c set -e if [[ $(uname -o) == "Darwin" ]]; then echo "\n[SAN]\nsubjectAltName=DNS:argocd-redis.$NAMESPACE.svc.cluster.local\n[req]\ndistinguished_name=req" >${PWD}/openssl_test.cnf else echo -e "\n[SAN]\nsubjectAltName=DNS:argocd-redis.$NAMESPACE.svc.cluster.local\n[req]\ndistinguished_name=req" >${PWD}/openssl_test.cnf fi openssl req -new -x509 \ -subj "/C=XX/ST=XX/O=Testing/CN=redis" \ -reqexts SAN -extensions SAN \ -config ${PWD}/openssl_test.cnf \ -keyout ${PWD}/redis.key \ -out ${PWD}/redis.crt \ -newkey rsa:4096 \ -nodes \ -sha256 \ -days 10 ] logger.go:42: 06:51:51 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | .+++++++++++++++++++++++++++++++++++++++++++++*.+.....+..........+............+...+.....+......+..........+..+...+.+...+..+......+.......+..+.+.....+.......+.....+...............+.+.....+.........+.+......+......+..............+......+.......+.....+...+...+.......+.....+.......+...+..+++++++++++++++++++++++++++++++++++++++++++++*....+.+..+......+.......+........+.......+...+..+...+....+..+..................................+.....+.......+........+......+................+..+...+...+.+.........+..+.......+......+..+...............+....+...............+.....+...+.+.....+......+.............+...+............+......+............+...+..+.............+.........+........+...+.........+......+.+........+.............+..+...+....+...+..................+..+...+.......+...............+......+..+.........+++++ logger.go:42: 06:51:54 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | ..+.+..+....+...+.....+......+.+......+............+..+.........+.+...........+...+.+........+.............+...............+..+.......+.....+...+.....................+..........+............+..+.+.........+..+.+..+......+....+........+.......+...+..+..........+............+...+...+.....+.......+...+..............+.+.....+...+.+........+.......+...........+...+....+.....+.+..+....+..............+++++++++++++++++++++++++++++++++++++++++++++*............+..+....+.....+..........+..+....+........+.......+...+++++++++++++++++++++++++++++++++++++++++++++*..+..........+..+.+..............+...+.........+.............+...+..............+..........+.........+......+........+...............+............+..........+........+..........+..+.+.................+...+....+..............+....+...+...+....................+..........+.....+....+........+...+.+.........+.........+........+....+.........+.........+......+..+....+...+... logger.go:42: 06:51:54 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | ----- I0220 06:51:55.315994 122 request.go:682] Waited for 1.031355552s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1?timeout=32s logger.go:42: 06:51:56 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | test step completed 2-generate_cert logger.go:42: 06:51:56 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | starting test step 3-create_secret logger.go:42: 06:51:56 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | running command: [sh -c set -e # clean up key, cert and config file cleanup() { rm -rf ${PWD}/redis.crt && rm -rf ${PWD}/redis.key && rm -rf ${PWD}/openssl_test.cnf } trap cleanup INT TERM EXIT oc create secret tls argocd-operator-redis-tls --key=${PWD}/redis.key --cert=${PWD}/redis.crt -n $NAMESPACE sleep 10 ] logger.go:42: 06:51:57 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | secret/argocd-operator-redis-tls created I0220 06:52:08.209728 122 request.go:682] Waited for 1.032094756s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 06:52:09 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | test step completed 3-create_secret logger.go:42: 06:52:09 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | starting test step 4-create_annotation logger.go:42: 06:52:09 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | running command: [sh -c set -e oc annotate secret argocd-operator-redis-tls argocds.argoproj.io/name=argocd -n $NAMESPACE sleep 30 ] logger.go:42: 06:52:10 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | secret/argocd-operator-redis-tls annotated I0220 06:52:30.086676 122 request.go:682] Waited for 1.032456355s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/argoproj.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 06:52:31 | 1-073_validate_rhsso/2-sleep | test step completed 2-sleep logger.go:42: 06:52:31 | 1-073_validate_rhsso/3-verify-oidc | starting test step 3-verify-oidc logger.go:42: 06:52:31 | 1-073_validate_rhsso/3-verify-oidc | running command: [sh -c issuer=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep issuer | awk -F' ' '{print $2}') keycloakRoute=$(kubectl get route keycloak -n $NAMESPACE -o jsonpath='{.spec.host}') if [[ "${issuer}" == "https://${keycloakRoute}/auth/realms/argocd" ]]; then echo "issuer matched" else echo "issuer mismatched" echo "${issuer} not equals ${keycloakRoute}/auth/realms/argocd" exit 1 fi ] logger.go:42: 06:52:32 | 1-073_validate_rhsso/3-verify-oidc | issuer matched logger.go:42: 06:52:32 | 1-073_validate_rhsso/3-verify-oidc | running command: [sh -c clientid=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep clientid | awk -F' ' '{print $2}') name=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep name | awk -F' ' '{print $2}') rootCA=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep rootca | awk -F' ' '{print $2}') if [[ "${clientid}" == "argocd" && "${name}" == "Keycloak" && "${rootCA}" == "'---BEGIN---END---'" ]]; then echo "oidc name, clientid and rootCA matched" else echo "oidc name, clientid or rootCA mismatched" echo "${clientid}, ${name} and ${rootCA}" exit 1 fi ] logger.go:42: 06:52:33 | 1-073_validate_rhsso/3-verify-oidc | oidc name, clientid and rootCA matched logger.go:42: 06:52:36 | 1-073_validate_rhsso/3-verify-oidc | test step completed 3-verify-oidc logger.go:42: 06:52:36 | 1-073_validate_rhsso/4-verifyRealmCreation | starting test step 4-verifyRealmCreation logger.go:42: 06:52:36 | 1-073_validate_rhsso/4-verifyRealmCreation | running command: [sh -c # Set the needed parameter for the authorization KEYCLOAK_URL=$(oc get route keycloak -n $NAMESPACE -o jsonpath='{.spec.host}') tenant=argocd USER=$(oc get secret keycloak-secret -n $NAMESPACE -o jsonpath='{.data.SSO_USERNAME}' | base64 --decode) PASSWORD=$(oc get secret keycloak-secret -n $NAMESPACE -o jsonpath='{.data.SSO_PASSWORD}' | base64 --decode) GRANT_TYPE=password CLIENT_ID=admin-cli # Execute the CURL command to request the access-token access_token=$(curl -d "client_id=$CLIENT_ID" -d "username=$USER" -d "password=$PASSWORD" -d "grant_type=$GRANT_TYPE" "https://$KEYCLOAK_URL/auth/realms/master/protocol/openid-connect/token" -k | sed -n 's|.*"access_token":"\([^"]*\)".*|\1|p') # Execute the CURL command to verify the realm and client creation clientFound=$(curl -H "Content-Type: application/json" -H "Authorization: bearer $access_token" "https://$KEYCLOAK_URL/auth/admin/realms/$tenant/clients" -k | grep '"clientId":"argocd"') if ! [ "$clientFound" = "" ]; then echo "argocd realm and client creation verified" else echo "argocd client not found" exit 1 fi # Verify OpenShift-v4 IdP creation IdPFound=$(curl -H "Content-Type: application/json" -H "Authorization: bearer $access_token" "https://$KEYCLOAK_URL/auth/admin/realms/$tenant/identity-provider/instances" -k | grep -i OpenShift-v4) if ! [ "$IdPFound" = "" ]; then echo "OpenShift-v4 IdP creation verified" else echo "OpenShift-v4 IdP not found" exit 1 fi # Verify OpenShift-v4 IdP creation syncModeVerified=$(curl -H "Content-Type: application/json" -H "Authorization: bearer $access_token" "https://$KEYCLOAK_URL/auth/admin/realms/$tenant/identity-provider/instances" -k | grep -i '"syncMode":"FORCE"') if ! [ "$syncModeVerified" = "" ]; then echo "syncMode set to Force, verified" else echo "syncMode not set to Force" exit 1 fi ] logger.go:42: 06:52:37 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 06:52:37 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 06:52:37 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0100 75 0 0 100 75 0 360 --:--:-- --:--:-- --:--:-- 358100 2153 100 2078 100 75 3935 142 --:--:-- --:--:-- --:--:-- 4069 logger.go:42: 06:52:37 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 06:52:37 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0100 6452 100 6452 0 0 10789 0 --:--:-- --:--:-- --:--:-- 10771100 6452 100 6452 0 0 10789 0 --:--:-- --:--:-- --:--:-- 10771 logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | argocd realm and client creation verified logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0100 557 100 557 0 0 6188 0 --:--:-- --:--:-- --:--:-- 6188 logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | OpenShift-v4 IdP creation verified logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0100 557 100 557 0 0 7736 0 --:--:-- --:--:-- --:--:-- 7736 logger.go:42: 06:52:38 | 1-073_validate_rhsso/4-verifyRealmCreation | syncMode set to Force, verified I0220 06:52:40.130073 122 request.go:682] Waited for 1.432489096s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/cloudcredential.openshift.io/v1?timeout=32s logger.go:42: 06:52:41 | 1-073_validate_rhsso/4-verifyRealmCreation | test step completed 4-verifyRealmCreation logger.go:42: 06:52:41 | 1-073_validate_rhsso/99-delete | starting test step 99-delete logger.go:42: 06:52:41 | 1-073_validate_rhsso/99-delete | running command: [oc delete argocd example-argocd-keycloak -n kuttl-test-helped-peacock] logger.go:42: 06:52:41 | 1-073_validate_rhsso/99-delete | argocd.argoproj.io "example-argocd-keycloak" deleted logger.go:42: 06:52:46 | 1-073_validate_rhsso/99-delete | test step completed 99-delete logger.go:42: 06:52:46 | 1-073_validate_rhsso | skipping kubernetes event logging logger.go:42: 06:52:46 | 1-073_validate_rhsso | Deleting namespace: kuttl-test-helped-peacock === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 06:52:46 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | test step completed 4-create_annotation logger.go:42: 06:52:46 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | starting test step 5-check_deployments logger.go:42: 06:52:46 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-redis -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.args}}{{"\n"}}{{end}}')" != \ "[redis-server \ --protected-mode no \ --save \ --appendonly no \ --requirepass \$(REDIS_PASSWORD) \ --tls-port 6379 \ --port 0 \ --tls-cert-file /app/config/redis/tls/tls.crt \ --tls-key-file /app/config/redis/tls/tls.key \ --tls-auth-clients no]"; then echo "TLS .spec.template.spec.containers.args for argocd-redis deployment are wrong" exit 1 fi ] logger.go:42: 06:52:46 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-repo-server -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[uid_entrypoint.sh \ argocd-repo-server \ --redis argocd-redis.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/reposerver/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-repo-server deployment is wrong" exit 1 fi ] logger.go:42: 06:52:47 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-server -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-server \ --staticassets /shared/app \ --dex-server https://argocd-dex-server.$NAMESPACE.svc.cluster.local:5556 \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --redis argocd-redis.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/server/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-server deployment is wrong" exit 1 fi ] logger.go:42: 06:52:50 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | test step completed 5-check_deployments logger.go:42: 06:52:50 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/6-check_statefulset | starting test step 6-check_statefulset logger.go:42: 06:52:50 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/6-check_statefulset | running command: [sh -c set -e if test "$(oc get statefulsets.apps argocd-application-controller -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-application-controller \ --operation-processors 10 \ --redis argocd-redis.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/controller/tls/redis/tls.crt \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --status-processors 20 \ --kubectl-parallelism-limit 10 \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-application-controller statefulsets is wrong" exit 1 fi ] I0220 06:52:51.567218 122 request.go:682] Waited for 1.033061301s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-071_validate_SCC_HA === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 06:52:53 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/6-check_statefulset | test step completed 6-check_statefulset logger.go:42: 06:52:53 | 1-066_validate_redis_secure_comm_no_autotls_no_ha | skipping kubernetes event logging === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 06:52:53 | 1-071_validate_SCC_HA | Creating namespace: kuttl-test-just-turtle === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 06:52:53 | 1-066_validate_redis_secure_comm_no_autotls_no_ha | Deleting namespace: kuttl-test-romantic-phoenix === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 06:52:53 | 1-071_validate_SCC_HA/1-create-scc | starting test step 1-create-scc === CONT kuttl/harness/1-065_validate_redis_ha_anti_affinity logger.go:42: 06:52:55 | 1-065_validate_redis_ha_anti_affinity/2- | test step completed 2- logger.go:42: 06:52:55 | 1-065_validate_redis_ha_anti_affinity | skipping kubernetes event logging === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 06:52:55 | 1-071_validate_SCC_HA/1-create-scc | SecurityContextConstraints:/restricted-dropcaps created === CONT kuttl/harness/1-065_validate_redis_ha_anti_affinity logger.go:42: 06:52:55 | 1-065_validate_redis_ha_anti_affinity | Deleting namespace: kuttl-test-internal-beetle === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 06:52:55 | 1-071_validate_SCC_HA/1-create-scc | test step completed 1-create-scc logger.go:42: 06:52:55 | 1-071_validate_SCC_HA/2-create-argocd | starting test step 2-create-argocd logger.go:42: 06:52:58 | 1-071_validate_SCC_HA/2-create-argocd | ArgoCD:kuttl-test-just-turtle/argocd created === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 06:52:59 | 1-070_validate_config_management_plugin | Creating namespace: kuttl-test-relevant-antelope logger.go:42: 06:52:59 | 1-070_validate_config_management_plugin/1-install | starting test step 1-install I0220 06:53:01.578799 122 request.go:682] Waited for 1.815656846s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s logger.go:42: 06:53:02 | 1-070_validate_config_management_plugin/1-install | Namespace:/argocd created logger.go:42: 06:53:02 | 1-070_validate_config_management_plugin/1-install | ConfigMap:argocd/cmp-plugin created logger.go:42: 06:53:02 | 1-070_validate_config_management_plugin/1-install | ArgoCD:argocd/argocd created === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 06:53:10 | 1-071_validate_SCC_HA/2-create-argocd | test step completed 2-create-argocd logger.go:42: 06:53:10 | 1-071_validate_SCC_HA/3-modify-argocd | starting test step 3-modify-argocd I0220 06:53:11.616554 122 request.go:682] Waited for 1.383268592s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s logger.go:42: 06:53:12 | 1-071_validate_SCC_HA/3-modify-argocd | ArgoCD:kuttl-test-just-turtle/argocd updated === CONT kuttl/harness/1-050_validate_sso logger.go:42: 06:53:22 | 1-050_validate_sso/8-remove-spec-dex | test step completed 8-remove-spec-dex logger.go:42: 06:53:22 | 1-050_validate_sso | skipping kubernetes event logging logger.go:42: 06:53:22 | 1-050_validate_sso | Deleting namespace: kuttl-test-main-squid === CONT kuttl/harness/1-103_argocd_alpha_to_beta_conversion logger.go:42: 06:53:34 | 1-103_argocd_alpha_to_beta_conversion | Creating namespace: kuttl-test-aware-crawdad logger.go:42: 06:53:34 | 1-103_argocd_alpha_to_beta_conversion/1-alpha_argocd | starting test step 1-alpha_argocd I0220 06:53:35.218630 122 request.go:682] Waited for 1.033657282s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 06:53:36 | 1-070_validate_config_management_plugin/1-install | test step completed 1-install logger.go:42: 06:53:36 | 1-070_validate_config_management_plugin/2-create-app | starting test step 2-create-app === CONT kuttl/harness/1-103_argocd_alpha_to_beta_conversion logger.go:42: 06:53:36 | 1-103_argocd_alpha_to_beta_conversion/1-alpha_argocd | ArgoCD:kuttl-test-aware-crawdad/argocd created === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 06:53:39 | 1-070_validate_config_management_plugin/2-create-app | Application:argocd/guestbook created === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 06:53:41 | 1-102_validate_handle_terminating_namespaces | Creating namespace: kuttl-test-welcome-blowfish logger.go:42: 06:53:41 | 1-102_validate_handle_terminating_namespaces/1-install | starting test step 1-install === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 06:53:43 | 1-070_validate_config_management_plugin/2-create-app | test step completed 2-create-app logger.go:42: 06:53:43 | 1-070_validate_config_management_plugin/99-delete | starting test step 99-delete === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 06:53:43 | 1-102_validate_handle_terminating_namespaces/1-install | Namespace:/gitops-service-argocd created logger.go:42: 06:53:43 | 1-102_validate_handle_terminating_namespaces/1-install | test step completed 1-install logger.go:42: 06:53:43 | 1-102_validate_handle_terminating_namespaces/2-install | starting test step 2-install I0220 06:53:45.250505 122 request.go:682] Waited for 1.481455872s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/cloud.network.openshift.io/v1?timeout=32s logger.go:42: 06:53:46 | 1-102_validate_handle_terminating_namespaces/2-install | ArgoCD:gitops-service-argocd/gitops-service-argocd created logger.go:42: 06:53:46 | 1-102_validate_handle_terminating_namespaces/2-install | Namespace:/jane created logger.go:42: 06:53:46 | 1-102_validate_handle_terminating_namespaces/2-install | ConfigMap:jane/my-config-map-2 created === CONT kuttl/harness/1-103_argocd_alpha_to_beta_conversion logger.go:42: 06:54:08 | 1-103_argocd_alpha_to_beta_conversion/1-alpha_argocd | test step completed 1-alpha_argocd logger.go:42: 06:54:08 | 1-103_argocd_alpha_to_beta_conversion/2-delete | starting test step 2-delete I0220 06:54:09.772082 122 request.go:682] Waited for 1.033009239s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apps.openshift.io/v1?timeout=32s logger.go:42: 06:54:11 | 1-103_argocd_alpha_to_beta_conversion/2-delete | test step completed 2-delete logger.go:42: 06:54:11 | 1-103_argocd_alpha_to_beta_conversion | skipping kubernetes event logging logger.go:42: 06:54:11 | 1-103_argocd_alpha_to_beta_conversion | Deleting namespace: kuttl-test-aware-crawdad === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:54:17 | 1-032_validate_dynamic_scaling | Creating namespace: kuttl-test-above-gull logger.go:42: 06:54:17 | 1-032_validate_dynamic_scaling/1-install | starting test step 1-install logger.go:42: 06:54:17 | 1-032_validate_dynamic_scaling/1-install | running command: [sleep 30s] === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 06:54:18 | 1-102_validate_handle_terminating_namespaces/2-install | test step completed 2-install logger.go:42: 06:54:18 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | starting test step 3-delete-ns logger.go:42: 06:54:18 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | running command: [kubectl delete ns jane] logger.go:42: 06:54:18 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | namespace "jane" deleted I0220 06:54:21.413761 122 request.go:682] Waited for 1.033288577s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1?timeout=32s === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 06:54:22 | 1-070_validate_config_management_plugin/99-delete | test step completed 99-delete logger.go:42: 06:54:22 | 1-070_validate_config_management_plugin | skipping kubernetes event logging logger.go:42: 06:54:22 | 1-070_validate_config_management_plugin | Deleting namespace: kuttl-test-relevant-antelope === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 06:54:28 | 1-047_validate_custom_env | Creating namespace: kuttl-test-oriented-ape logger.go:42: 06:54:28 | 1-047_validate_custom_env/1-install | starting test step 1-install logger.go:42: 06:54:31 | 1-047_validate_custom_env/1-install | ArgoCD:kuttl-test-oriented-ape/argocd created I0220 06:54:48.570164 122 request.go:682] Waited for 1.033224009s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:54:50 | 1-032_validate_dynamic_scaling/1-install | ArgoCD:kuttl-test-above-gull/argocd created === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 06:54:52 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | test step completed 3-delete-ns logger.go:42: 06:54:52 | 1-102_validate_handle_terminating_namespaces/4-create-ns | starting test step 4-create-ns logger.go:42: 06:54:55 | 1-102_validate_handle_terminating_namespaces/4-create-ns | Namespace:/john created logger.go:42: 06:54:55 | 1-102_validate_handle_terminating_namespaces/4-create-ns | test step completed 4-create-ns logger.go:42: 06:54:55 | 1-102_validate_handle_terminating_namespaces/5-create-app | starting test step 5-create-app logger.go:42: 06:54:57 | 1-102_validate_handle_terminating_namespaces/5-create-app | Application:gitops-service-argocd/my-app created logger.go:42: 06:55:00 | 1-102_validate_handle_terminating_namespaces/5-create-app | test step completed 5-create-app logger.go:42: 06:55:00 | 1-102_validate_handle_terminating_namespaces/99-delete | starting test step 99-delete === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 06:55:02 | 1-047_validate_custom_env/1-install | test step completed 1-install logger.go:42: 06:55:02 | 1-047_validate_custom_env/2-change-env-vars | starting test step 2-change-env-vars I0220 06:55:03.767651 122 request.go:682] Waited for 1.032537189s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/ingress.operator.openshift.io/v1?timeout=32s logger.go:42: 06:55:05 | 1-047_validate_custom_env/2-change-env-vars | ArgoCD:kuttl-test-oriented-ape/argocd updated logger.go:42: 06:55:05 | 1-047_validate_custom_env/2-change-env-vars | test step completed 2-change-env-vars logger.go:42: 06:55:05 | 1-047_validate_custom_env/3-check-workloads | starting test step 3-check-workloads logger.go:42: 06:55:05 | 1-047_validate_custom_env/3-check-workloads | running command: [sh -c sleep 10] === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 06:55:13 | 1-102_validate_handle_terminating_namespaces/99-delete | running command: [sh -c oc patch cm/my-config-map-2 -n jane --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" }]'] ] logger.go:42: 06:55:13 | 1-102_validate_handle_terminating_namespaces/99-delete | configmap/my-config-map-2 patched I0220 06:55:14.616399 122 request.go:682] Waited for 1.028420563s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 06:55:15 | 1-047_validate_custom_env/3-check-workloads | running command: [sh -c set -e set -o pipefail for wl in deployment/argocd-server deployment/argocd-repo-server statefulset/argocd-application-controller; do val=$(oc get -n $NAMESPACE $wl -o json | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="FOO").value') if test "$val" != "bar"; then echo "Environment for $wl was not set correctly." exit 1 fi done ] === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 06:55:16 | 1-102_validate_handle_terminating_namespaces/99-delete | test step completed 99-delete logger.go:42: 06:55:16 | 1-102_validate_handle_terminating_namespaces | skipping kubernetes event logging logger.go:42: 06:55:16 | 1-102_validate_handle_terminating_namespaces | Deleting namespace: kuttl-test-welcome-blowfish === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 06:55:19 | 1-047_validate_custom_env/3-check-workloads | test step completed 3-check-workloads logger.go:42: 06:55:19 | 1-047_validate_custom_env | skipping kubernetes event logging logger.go:42: 06:55:19 | 1-047_validate_custom_env | Deleting namespace: kuttl-test-oriented-ape === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:55:21 | 1-032_validate_dynamic_scaling/1-install | test step completed 1-install logger.go:42: 06:55:21 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | starting test step 2-add-cluster-secrets logger.go:42: 06:55:21 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | running command: [sleep 30s] === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 06:55:22 | 1-045_validate_repo_exec_timeout | Creating namespace: kuttl-test-ace-bison logger.go:42: 06:55:22 | 1-045_validate_repo_exec_timeout/1-install | starting test step 1-install I0220 06:55:24.639946 122 request.go:682] Waited for 2.433014327s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/coordination.k8s.io/v1?timeout=32s logger.go:42: 06:55:24 | 1-045_validate_repo_exec_timeout/1-install | ArgoCD:kuttl-test-ace-bison/argocd created === CONT kuttl/harness/1-038_validate_productized_images logger.go:42: 06:55:25 | 1-038_validate_productized_images | Creating namespace: kuttl-test-frank-dove logger.go:42: 06:55:25 | 1-038_validate_productized_images/1-install | starting test step 1-install logger.go:42: 06:55:28 | 1-038_validate_productized_images/1-install | ArgoCD:kuttl-test-frank-dove/argocd created I0220 06:55:52.470321 122 request.go:682] Waited for 1.033047079s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:55:53 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | Secret:kuttl-test-above-gull/cluster-1 created logger.go:42: 06:55:53 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | Secret:kuttl-test-above-gull/cluster-2 created === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 06:55:55 | 1-045_validate_repo_exec_timeout/1-install | test step completed 1-install logger.go:42: 06:55:55 | 1-045_validate_repo_exec_timeout/2-change-exec-timeout | starting test step 2-change-exec-timeout logger.go:42: 06:55:58 | 1-045_validate_repo_exec_timeout/2-change-exec-timeout | ArgoCD:kuttl-test-ace-bison/argocd updated logger.go:42: 06:55:58 | 1-045_validate_repo_exec_timeout/2-change-exec-timeout | test step completed 2-change-exec-timeout logger.go:42: 06:55:58 | 1-045_validate_repo_exec_timeout/4-check-workload-env | starting test step 4-check-workload-env logger.go:42: 06:55:58 | 1-045_validate_repo_exec_timeout/4-check-workload-env | running command: [sh -c sleep 10] === CONT kuttl/harness/1-038_validate_productized_images logger.go:42: 06:56:00 | 1-038_validate_productized_images/1-install | test step completed 1-install logger.go:42: 06:56:00 | 1-038_validate_productized_images/2-check-images | starting test step 2-check-images logger.go:42: 06:56:00 | 1-038_validate_productized_images/2-check-images | running command: [sh -c for wl in deployment/argocd-server deployment/argocd-repo-server statefulset/argocd-application-controller; do image=$(oc -n $NAMESPACE get ${wl} -o jsonpath='{.spec.template.spec.containers[0].image}' | awk -F'@' '{print $1}') if test "$image" != "registry.redhat.io/openshift-gitops-1/argocd-rhel8"; then echo "Non-productized image in workload $wl detected." exit 1 fi done ] I0220 06:56:02.961935 122 request.go:682] Waited for 1.031797114s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/admissionregistration.k8s.io/v1?timeout=32s logger.go:42: 06:56:04 | 1-038_validate_productized_images/2-check-images | test step completed 2-check-images logger.go:42: 06:56:04 | 1-038_validate_productized_images | skipping kubernetes event logging logger.go:42: 06:56:04 | 1-038_validate_productized_images | Deleting namespace: kuttl-test-frank-dove === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 06:56:08 | 1-045_validate_repo_exec_timeout/4-check-workload-env | running command: [sh -c timeout=$(oc get -n $NAMESPACE deployment argocd-repo-server -o json \ | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="ARGOCD_EXEC_TIMEOUT").value') if test "$timeout" != "300s"; then echo "Assertion failed. Timeout should be 300, is '$timeout'" exit 1 fi ] === CONT kuttl/harness/1-045_validate_applicationset_image_override === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 06:56:11 | 1-045_validate_repo_exec_timeout/4-check-workload-env | test step completed 4-check-workload-env logger.go:42: 06:56:11 | 1-045_validate_repo_exec_timeout | skipping kubernetes event logging === CONT kuttl/harness/1-045_validate_applicationset_image_override logger.go:42: 06:56:11 | 1-045_validate_applicationset_image_override | Creating namespace: kuttl-test-superb-seal logger.go:42: 06:56:11 | 1-045_validate_applicationset_image_override/1-add_applicationset_image | starting test step 1-add_applicationset_image === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 06:56:11 | 1-045_validate_repo_exec_timeout | Deleting namespace: kuttl-test-ace-bison I0220 06:56:12.997070 122 request.go:682] Waited for 1.031974815s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/authorization.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-045_validate_applicationset_image_override logger.go:42: 06:56:14 | 1-045_validate_applicationset_image_override/1-add_applicationset_image | ArgoCD:kuttl-test-superb-seal/example-argocd created logger.go:42: 06:56:15 | 1-045_validate_applicationset_image_override/1-add_applicationset_image | test step completed 1-add_applicationset_image logger.go:42: 06:56:15 | 1-045_validate_applicationset_image_override/2-remove_argocdcustom_image | starting test step 2-remove_argocdcustom_image === CONT kuttl/harness/1-044_validate_resource_limit_changes logger.go:42: 06:56:18 | 1-044_validate_resource_limit_changes | Creating namespace: kuttl-test-cool-mouse logger.go:42: 06:56:19 | 1-044_validate_resource_limit_changes/1-install | starting test step 1-install === CONT kuttl/harness/1-045_validate_applicationset_image_override logger.go:42: 06:56:19 | 1-045_validate_applicationset_image_override/2-remove_argocdcustom_image | ArgoCD:kuttl-test-superb-seal/example-argocd updated logger.go:42: 06:56:22 | 1-045_validate_applicationset_image_override/2-remove_argocdcustom_image | test step completed 2-remove_argocdcustom_image logger.go:42: 06:56:22 | 1-045_validate_applicationset_image_override | skipping kubernetes event logging === CONT kuttl/harness/1-044_validate_resource_limit_changes logger.go:42: 06:56:22 | 1-044_validate_resource_limit_changes/1-install | ArgoCD:kuttl-test-cool-mouse/argocd created === CONT kuttl/harness/1-045_validate_applicationset_image_override logger.go:42: 06:56:22 | 1-045_validate_applicationset_image_override | Deleting namespace: kuttl-test-superb-seal === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:56:26 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | test step completed 2-add-cluster-secrets logger.go:42: 06:56:26 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | starting test step 3-cluster-secrets-exceed-maxShards logger.go:42: 06:56:26 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sleep 60s] === CONT kuttl/harness/1-033_validate_resource_exclusions logger.go:42: 06:56:28 | 1-033_validate_resource_exclusions | Creating namespace: kuttl-test-gentle-bird logger.go:42: 06:56:28 | 1-033_validate_resource_exclusions/1-install | starting test step 1-install I0220 06:56:29.694800 122 request.go:682] Waited for 1.032133536s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1beta1?timeout=32s logger.go:42: 06:56:31 | 1-033_validate_resource_exclusions/1-install | ArgoCD:kuttl-test-gentle-bird/argocd created logger.go:42: 06:56:42 | 1-033_validate_resource_exclusions/1-install | test step completed 1-install logger.go:42: 06:56:42 | 1-033_validate_resource_exclusions/2-add_resource_exclusions | starting test step 2-add_resource_exclusions I0220 06:56:43.664541 122 request.go:682] Waited for 1.032125841s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/authorization.k8s.io/v1?timeout=32s logger.go:42: 06:56:45 | 1-033_validate_resource_exclusions/2-add_resource_exclusions | ArgoCD:kuttl-test-gentle-bird/argocd updated logger.go:42: 06:56:46 | 1-033_validate_resource_exclusions/2-add_resource_exclusions | test step completed 2-add_resource_exclusions logger.go:42: 06:56:46 | 1-033_validate_resource_exclusions | skipping kubernetes event logging logger.go:42: 06:56:46 | 1-033_validate_resource_exclusions | Deleting namespace: kuttl-test-gentle-bird === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 06:56:52 | 1-043_validate_log_level_format | Creating namespace: kuttl-test-blessed-yeti logger.go:42: 06:56:52 | 1-043_validate_log_level_format/1-install | starting test step 1-install I0220 06:56:53.697931 122 request.go:682] Waited for 1.28271132s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha2?timeout=32s === CONT kuttl/harness/1-044_validate_resource_limit_changes logger.go:42: 06:56:54 | 1-044_validate_resource_limit_changes/1-install | test step completed 1-install logger.go:42: 06:56:54 | 1-044_validate_resource_limit_changes/2-change-limits | starting test step 2-change-limits === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 06:56:54 | 1-043_validate_log_level_format/1-install | ArgoCD:kuttl-test-blessed-yeti/argocd created === CONT kuttl/harness/1-044_validate_resource_limit_changes logger.go:42: 06:56:57 | 1-044_validate_resource_limit_changes/2-change-limits | ArgoCD:kuttl-test-cool-mouse/argocd updated logger.go:42: 06:56:57 | 1-044_validate_resource_limit_changes/2-change-limits | test step completed 2-change-limits logger.go:42: 06:56:57 | 1-044_validate_resource_limit_changes/4-check-workloads | starting test step 4-check-workloads logger.go:42: 06:56:57 | 1-044_validate_resource_limit_changes/4-check-workloads | running command: [sh -c sleep 10] logger.go:42: 06:57:07 | 1-044_validate_resource_limit_changes/4-check-workloads | running command: [sh -c set -e set -o pipefail for wl in deployments/argocd-server deployments/argocd-repo-server statefulsets/argocd-application-controller; do res=$(oc get -n $NAMESPACE $wl -o jsonpath='{.spec.template.spec.containers[0].resources.limits.cpu}') if test "$res" != "2"; then echo "Reconciliation of resources for $wl failed: Should be 2, is '$res'" exit 1 fi done ] I0220 06:57:09.912378 122 request.go:682] Waited for 1.02935833s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/migration.k8s.io/v1alpha1?timeout=32s logger.go:42: 06:57:11 | 1-044_validate_resource_limit_changes/4-check-workloads | test step completed 4-check-workloads logger.go:42: 06:57:11 | 1-044_validate_resource_limit_changes | skipping kubernetes event logging logger.go:42: 06:57:11 | 1-044_validate_resource_limit_changes | Deleting namespace: kuttl-test-cool-mouse === CONT kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount logger.go:42: 06:57:17 | 1-033_validate_applicationset_tls_scm_volume_mount | Creating namespace: kuttl-test-upright-oyster logger.go:42: 06:57:17 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | starting test step 1-install I0220 06:57:19.952317 122 request.go:682] Waited for 2.278376258s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1?timeout=32s logger.go:42: 06:57:20 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | Namespace:/test-1-32-appsets-scm-tls-mount created logger.go:42: 06:57:20 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | ConfigMap:test-1-32-appsets-scm-tls-mount/argocd-appset-gitlab-scm-tls-certs-cm created logger.go:42: 06:57:20 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | ArgoCD:test-1-32-appsets-scm-tls-mount/argocd created === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 06:57:29 | 1-043_validate_log_level_format/1-install | test step completed 1-install logger.go:42: 06:57:29 | 1-043_validate_log_level_format/2-change-loglevel | starting test step 2-change-loglevel === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:57:29 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-above-gull/cluster-3 created logger.go:42: 06:57:29 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-above-gull/cluster-4 created logger.go:42: 06:57:29 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-above-gull/cluster-5 created logger.go:42: 06:57:29 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-above-gull/cluster-6 created I0220 06:57:30.050674 122 request.go:682] Waited for 1.030013052s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 06:57:31 | 1-043_validate_log_level_format/2-change-loglevel | ArgoCD:kuttl-test-blessed-yeti/argocd updated logger.go:42: 06:57:31 | 1-043_validate_log_level_format/2-change-loglevel | test step completed 2-change-loglevel logger.go:42: 06:57:31 | 1-043_validate_log_level_format/3-check-loglevel | starting test step 3-check-loglevel logger.go:42: 06:57:31 | 1-043_validate_log_level_format/3-check-loglevel | running command: [sh -c sleep 15] === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:57:31 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 06:57:32 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 06:57:33 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 06:57:33 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 06:57:34 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 06:57:34 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 06:57:35 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 06:57:36 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 06:57:37 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 06:57:37 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 06:57:38 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 06:57:39 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 06:57:40 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 06:57:40 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 06:57:40 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | test step completed 3-cluster-secrets-exceed-maxShards logger.go:42: 06:57:40 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | starting test step 4-delete-cluster-secret logger.go:42: 06:57:40 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | running command: [sleep 60s] === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 06:57:46 | 1-043_validate_log_level_format/3-check-loglevel | running command: [sh -c set -e set -o pipefail for wl in deployment/argocd-server deployment/argocd-repo-server statefulset/argocd-application-controller; do wlCommand=$(oc get -n $NAMESPACE $wl -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) level="debug" format="json" if ! echo "$wlCommand" | grep -qPz -- "--loglevel\\n$level(\$|\\n)"; then echo "logLevel was not set correctly for $wl" echo "CWD: $wlCommand" exit 1 fi if ! echo "$wlCommand" | grep -qPz -- "--logformat\\n$format(\$|\\n)"; then echo "logFormat was not set correctly for $wl" echo "CWD: $wlCommand" exit 1 fi done ] I0220 06:57:49.261404 122 request.go:682] Waited for 1.032798649s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/build.openshift.io/v1?timeout=32s logger.go:42: 06:57:50 | 1-043_validate_log_level_format/3-check-loglevel | test step completed 3-check-loglevel logger.go:42: 06:57:50 | 1-043_validate_log_level_format | skipping kubernetes event logging logger.go:42: 06:57:50 | 1-043_validate_log_level_format | Deleting namespace: kuttl-test-blessed-yeti === CONT kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount logger.go:42: 06:57:52 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | test step completed 1-install logger.go:42: 06:57:52 | 1-033_validate_applicationset_tls_scm_volume_mount | skipping kubernetes event logging logger.go:42: 06:57:52 | 1-033_validate_applicationset_tls_scm_volume_mount | Deleting namespace: kuttl-test-upright-oyster === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:57:56 | 1-049_validate_parallelism_limit | Creating namespace: kuttl-test-pretty-giraffe logger.go:42: 06:57:56 | 1-049_validate_parallelism_limit/1-install | starting test step 1-install === CONT kuttl/harness/1-032_validate_resource_inclusions I0220 06:57:59.308343 122 request.go:682] Waited for 2.382532103s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/node.k8s.io/v1?timeout=32s logger.go:42: 06:57:59 | 1-032_validate_resource_inclusions | Creating namespace: kuttl-test-prepared-dinosaur logger.go:42: 06:57:59 | 1-032_validate_resource_inclusions/1-install | starting test step 1-install === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:57:59 | 1-049_validate_parallelism_limit/1-install | ArgoCD:kuttl-test-pretty-giraffe/argocd created === CONT kuttl/harness/1-032_validate_resource_inclusions logger.go:42: 06:58:01 | 1-032_validate_resource_inclusions/1-install | ArgoCD:kuttl-test-prepared-dinosaur/argocd created === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:11 | 1-049_validate_parallelism_limit/1-install | test step completed 1-install logger.go:42: 06:58:11 | 1-049_validate_parallelism_limit/2-check-deployment | starting test step 2-check-deployment logger.go:42: 06:58:11 | 1-049_validate_parallelism_limit/2-check-deployment | running command: [sh -c set -e set -o pipefail expected=10 wlCommand=$(oc get -n $NAMESPACE statefulset/argocd-application-controller -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) if ! echo "$wlCommand" | grep -qPz -- "--kubectl-parallelism-limit\\n${expected}(\$|\\n)"; then echo "Incorrect or missing --kubectl-parallelism-limit detected." echo "$wlCommand" exit 1 fi ] I0220 06:58:13.230903 122 request.go:682] Waited for 1.031268477s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/route.openshift.io/v1?timeout=32s logger.go:42: 06:58:14 | 1-049_validate_parallelism_limit/2-check-deployment | test step completed 2-check-deployment logger.go:42: 06:58:14 | 1-049_validate_parallelism_limit/3-change-limit | starting test step 3-change-limit === CONT kuttl/harness/1-032_validate_resource_inclusions logger.go:42: 06:58:17 | 1-032_validate_resource_inclusions/1-install | test step completed 1-install logger.go:42: 06:58:17 | 1-032_validate_resource_inclusions/2-add_resource_inclusions | starting test step 2-add_resource_inclusions === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:17 | 1-049_validate_parallelism_limit/3-change-limit | ArgoCD:kuttl-test-pretty-giraffe/argocd updated logger.go:42: 06:58:17 | 1-049_validate_parallelism_limit/3-change-limit | test step completed 3-change-limit logger.go:42: 06:58:17 | 1-049_validate_parallelism_limit/4-check-deployment | starting test step 4-check-deployment logger.go:42: 06:58:19 | 1-049_validate_parallelism_limit/4-check-deployment | running command: [sh -c sleep 5] === CONT kuttl/harness/1-032_validate_resource_inclusions logger.go:42: 06:58:19 | 1-032_validate_resource_inclusions/2-add_resource_inclusions | ArgoCD:kuttl-test-prepared-dinosaur/argocd updated logger.go:42: 06:58:20 | 1-032_validate_resource_inclusions/2-add_resource_inclusions | test step completed 2-add_resource_inclusions logger.go:42: 06:58:20 | 1-032_validate_resource_inclusions | skipping kubernetes event logging logger.go:42: 06:58:20 | 1-032_validate_resource_inclusions | Deleting namespace: kuttl-test-prepared-dinosaur === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:24 | 1-049_validate_parallelism_limit/4-check-deployment | running command: [sh -c set -e set -o pipefail expected=20 wlCommand=$(oc get -n $NAMESPACE statefulset/argocd-application-controller -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) if ! echo "$wlCommand" | grep -qPz -- "--kubectl-parallelism-limit\\n${expected}(\$|\\n)"; then echo "Incorrect or missing --kubectl-parallelism-limit detected." echo "$wlCommand" exit 1 fi ] I0220 06:58:26.118020 122 request.go:682] Waited for 1.032315277s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/project.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-043_validate_custom_labels_annotations === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:27 | 1-049_validate_parallelism_limit/4-check-deployment | test step completed 4-check-deployment logger.go:42: 06:58:27 | 1-049_validate_parallelism_limit/5-change-back-to-default | starting test step 5-change-back-to-default === CONT kuttl/harness/1-043_validate_custom_labels_annotations logger.go:42: 06:58:30 | 1-043_validate_custom_labels_annotations | Creating namespace: kuttl-test-composed-mite logger.go:42: 06:58:30 | 1-043_validate_custom_labels_annotations/1-install | starting test step 1-install === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:30 | 1-049_validate_parallelism_limit/5-change-back-to-default | ArgoCD:kuttl-test-pretty-giraffe/argocd updated logger.go:42: 06:58:30 | 1-049_validate_parallelism_limit/5-change-back-to-default | test step completed 5-change-back-to-default logger.go:42: 06:58:30 | 1-049_validate_parallelism_limit/6-check-deployment | starting test step 6-check-deployment logger.go:42: 06:58:32 | 1-049_validate_parallelism_limit/6-check-deployment | running command: [sh -c set -e oc patch -n $NAMESPACE argocds/argocd --type=json --patch '[{"op": "remove", "path": "/spec/controller/parallelismLimit"}]' ] === CONT kuttl/harness/1-043_validate_custom_labels_annotations logger.go:42: 06:58:32 | 1-043_validate_custom_labels_annotations/1-install | ArgoCD:kuttl-test-composed-mite/argocd-sample created === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:33 | 1-049_validate_parallelism_limit/6-check-deployment | argocd.argoproj.io/argocd patched logger.go:42: 06:58:33 | 1-049_validate_parallelism_limit/6-check-deployment | running command: [sh -c sleep 5] === CONT kuttl/harness/1-043_validate_custom_labels_annotations logger.go:42: 06:58:33 | 1-043_validate_custom_labels_annotations/1-install | test step completed 1-install logger.go:42: 06:58:33 | 1-043_validate_custom_labels_annotations/2-delete-labels-annotations | starting test step 2-delete-labels-annotations I0220 06:58:36.118312 122 request.go:682] Waited for 2.283241526s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta3?timeout=32s logger.go:42: 06:58:36 | 1-043_validate_custom_labels_annotations/2-delete-labels-annotations | ArgoCD:kuttl-test-composed-mite/argocd-sample updated logger.go:42: 06:58:36 | 1-043_validate_custom_labels_annotations/2-delete-labels-annotations | test step completed 2-delete-labels-annotations logger.go:42: 06:58:36 | 1-043_validate_custom_labels_annotations | skipping kubernetes event logging logger.go:42: 06:58:36 | 1-043_validate_custom_labels_annotations | Deleting namespace: kuttl-test-composed-mite === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:38 | 1-049_validate_parallelism_limit/6-check-deployment | running command: [sh -c set -e set -o pipefail expected=10 wlCommand=$(oc get -n $NAMESPACE statefulset/argocd-application-controller -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) if ! echo "$wlCommand" | grep -qPz -- "--kubectl-parallelism-limit\\n${expected}(\$|\\n)"; then echo "Incorrect or missing --kubectl-parallelism-limit detected." echo "$wlCommand" exit 1 fi ] === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:58:40 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | running command: [kubectl delete Secret cluster-2 cluster-3 cluster-4 cluster-5 cluster-6 -n kuttl-test-above-gull] === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 06:58:40 | 1-049_validate_parallelism_limit/6-check-deployment | test step completed 6-check-deployment logger.go:42: 06:58:40 | 1-049_validate_parallelism_limit | skipping kubernetes event logging logger.go:42: 06:58:40 | 1-049_validate_parallelism_limit | Deleting namespace: kuttl-test-pretty-giraffe === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:58:41 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-2" deleted logger.go:42: 06:58:41 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-3" deleted logger.go:42: 06:58:41 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-4" deleted logger.go:42: 06:58:41 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-5" deleted logger.go:42: 06:58:41 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-6" deleted logger.go:42: 06:58:41 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | running command: [sleep 30s] === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 06:58:42 | 1-037_validate_argocd_setting_replicas | Creating namespace: kuttl-test-alert-dog logger.go:42: 06:58:42 | 1-037_validate_argocd_setting_replicas/1-install | starting test step 1-install logger.go:42: 06:58:45 | 1-037_validate_argocd_setting_replicas/1-install | ArgoCD:kuttl-test-alert-dog/argocd created === CONT kuttl/harness/1-042_restricted_pss_compliant 2025/02/20 06:58:47 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/parallel/1-042_restricted_pss_compliant/01-install-argocd-in-restricted-pss-ns copy.yaml logger.go:42: 06:58:47 | 1-042_restricted_pss_compliant | Creating namespace: kuttl-test-dear-mako logger.go:42: 06:58:47 | 1-042_restricted_pss_compliant/1-install-argocd-in-restricted-pss-ns copy | starting test step 1-install-argocd-in-restricted-pss-ns copy I0220 06:58:48.089311 122 request.go:682] Waited for 1.031356024s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1beta1?timeout=32s logger.go:42: 06:58:49 | 1-042_restricted_pss_compliant/1-install-argocd-in-restricted-pss-ns copy | Namespace:/test-1-042-restricted-pss-compliant created logger.go:42: 06:58:49 | 1-042_restricted_pss_compliant/1-install-argocd-in-restricted-pss-ns copy | ArgoCD:test-1-042-restricted-pss-compliant/argocd created === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 06:58:56 | 1-037_validate_argocd_setting_replicas/1-install | test step completed 1-install logger.go:42: 06:58:56 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | starting test step 2-scale_out_server logger.go:42: 06:58:56 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/server/replicas", "value": 3 }]' exit 0 ] logger.go:42: 06:58:57 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | argocd.argoproj.io/argocd patched I0220 06:58:58.214054 122 request.go:682] Waited for 1.033474031s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s logger.go:42: 06:58:59 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | test step completed 2-scale_out_server logger.go:42: 06:58:59 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | starting test step 3-scale_out_repo logger.go:42: 06:58:59 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/repo/replicas", "value": 3 }]' exit 0 ] logger.go:42: 06:59:00 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | argocd.argoproj.io/argocd patched logger.go:42: 06:59:02 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | test step completed 3-scale_out_repo logger.go:42: 06:59:02 | 1-037_validate_argocd_setting_replicas/4- | starting test step 4- I0220 06:59:12.207814 122 request.go:682] Waited for 1.032813891s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/config.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 06:59:13 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | test step completed 4-delete-cluster-secret logger.go:42: 06:59:13 | 1-032_validate_dynamic_scaling | skipping kubernetes event logging logger.go:42: 06:59:14 | 1-032_validate_dynamic_scaling | Deleting namespace: kuttl-test-above-gull === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 06:59:20 | 1-034_validate_webhook_notifications | Creating namespace: kuttl-test-real-werewolf logger.go:42: 06:59:20 | 1-034_validate_webhook_notifications/0-setup-webhook-server | starting test step 0-setup-webhook-server I0220 06:59:22.221843 122 request.go:682] Waited for 1.833218004s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1?timeout=32s logger.go:42: 06:59:22 | 1-034_validate_webhook_notifications/0-setup-webhook-server | Service:kuttl-test-real-werewolf/webhook created logger.go:42: 06:59:22 | 1-034_validate_webhook_notifications/0-setup-webhook-server | Deployment:kuttl-test-real-werewolf/webhook created logger.go:42: 06:59:23 | 1-034_validate_webhook_notifications/0-setup-webhook-server | ConfigMap:kuttl-test-real-werewolf/webhook-config created logger.go:42: 06:59:23 | 1-034_validate_webhook_notifications/0-setup-webhook-server | Secret:kuttl-test-real-werewolf/webhook-tls created === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 06:59:28 | 1-037_validate_argocd_setting_replicas/4- | test step completed 4- logger.go:42: 06:59:28 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | starting test step 5-scale_in_server logger.go:42: 06:59:28 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/server/replicas", "value": 1 }]' exit 0 ] logger.go:42: 06:59:28 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | argocd.argoproj.io/argocd patched logger.go:42: 06:59:30 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | test step completed 5-scale_in_server logger.go:42: 06:59:30 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | starting test step 6-scale_in_repo logger.go:42: 06:59:30 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/repo/replicas", "value": 1 }]' exit 0 ] === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 06:59:30 | 1-034_validate_webhook_notifications/0-setup-webhook-server | test step completed 0-setup-webhook-server logger.go:42: 06:59:30 | 1-034_validate_webhook_notifications/1-install-argocd | starting test step 1-install-argocd === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 06:59:31 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | argocd.argoproj.io/argocd patched I0220 06:59:32.259096 122 request.go:682] Waited for 1.283286098s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 06:59:33 | 1-034_validate_webhook_notifications/1-install-argocd | ArgoCD:kuttl-test-real-werewolf/argocd created === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 06:59:35 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | test step completed 6-scale_in_repo logger.go:42: 06:59:35 | 1-037_validate_argocd_setting_replicas/7- | starting test step 7- logger.go:42: 06:59:38 | 1-037_validate_argocd_setting_replicas/7- | test step completed 7- logger.go:42: 06:59:38 | 1-037_validate_argocd_setting_replicas | skipping kubernetes event logging === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 06:59:38 | 1-069_validate_redis_secure_comm_autotls_ha/1-install | test step completed 1-install logger.go:42: 06:59:38 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | starting test step 2-enable_autotls logger.go:42: 06:59:38 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"redis":{"autotls":"openshift"}}}' -n $NAMESPACE ] === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 06:59:38 | 1-037_validate_argocd_setting_replicas | Deleting namespace: kuttl-test-alert-dog === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 06:59:39 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | argocd.argoproj.io/argocd patched === CONT kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm logger.go:42: 06:59:44 | 1-039_validate_fix_argocd-tls-certs-cm | Creating namespace: kuttl-test-neutral-toucan logger.go:42: 06:59:44 | 1-039_validate_fix_argocd-tls-certs-cm/1-install | starting test step 1-install I0220 06:59:45.971812 122 request.go:682] Waited for 1.029556211s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/authentication.k8s.io/v1?timeout=32s logger.go:42: 06:59:47 | 1-039_validate_fix_argocd-tls-certs-cm/1-install | ArgoCD:kuttl-test-neutral-toucan/example-argocd created === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 07:00:04 | 1-034_validate_webhook_notifications/1-install-argocd | test step completed 1-install-argocd logger.go:42: 07:00:04 | 1-034_validate_webhook_notifications/2-add-notification-confs | starting test step 2-add-notification-confs I0220 07:00:05.840698 122 request.go:682] Waited for 1.033844554s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s logger.go:42: 07:00:07 | 1-034_validate_webhook_notifications/2-add-notification-confs | NotificationsConfiguration:kuttl-test-real-werewolf/default-notifications-configuration updated logger.go:42: 07:00:07 | 1-034_validate_webhook_notifications/2-add-notification-confs | test step completed 2-add-notification-confs logger.go:42: 07:00:07 | 1-034_validate_webhook_notifications/3-sleep | starting test step 3-sleep logger.go:42: 07:00:07 | 1-034_validate_webhook_notifications/3-sleep | running command: [sh -c sleep 20] === CONT kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm logger.go:42: 07:00:18 | 1-039_validate_fix_argocd-tls-certs-cm/1-install | test step completed 1-install logger.go:42: 07:00:18 | 1-039_validate_fix_argocd-tls-certs-cm/2-modify_configmap | starting test step 2-modify_configmap I0220 07:00:19.872064 122 request.go:682] Waited for 1.031434364s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s logger.go:42: 07:00:21 | 1-039_validate_fix_argocd-tls-certs-cm/2-modify_configmap | ConfigMap:kuttl-test-neutral-toucan/argocd-tls-certs-cm updated logger.go:42: 07:00:21 | 1-039_validate_fix_argocd-tls-certs-cm/2-modify_configmap | test step completed 2-modify_configmap logger.go:42: 07:00:21 | 1-039_validate_fix_argocd-tls-certs-cm/3-modify_argocd_cr | starting test step 3-modify_argocd_cr logger.go:42: 07:00:23 | 1-039_validate_fix_argocd-tls-certs-cm/3-modify_argocd_cr | ArgoCD:kuttl-test-neutral-toucan/example-argocd updated logger.go:42: 07:00:23 | 1-039_validate_fix_argocd-tls-certs-cm/3-modify_argocd_cr | test step completed 3-modify_argocd_cr logger.go:42: 07:00:23 | 1-039_validate_fix_argocd-tls-certs-cm/4- | starting test step 4- logger.go:42: 07:00:26 | 1-039_validate_fix_argocd-tls-certs-cm/4- | test step completed 4- logger.go:42: 07:00:26 | 1-039_validate_fix_argocd-tls-certs-cm | skipping kubernetes event logging logger.go:42: 07:00:26 | 1-039_validate_fix_argocd-tls-certs-cm | Deleting namespace: kuttl-test-neutral-toucan I0220 07:00:29.882146 122 request.go:682] Waited for 2.481526251s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/networking.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 07:00:29 | 1-034_validate_webhook_notifications/3-sleep | test step completed 3-sleep logger.go:42: 07:00:29 | 1-034_validate_webhook_notifications/4-create-app | starting test step 4-create-app logger.go:42: 07:00:29 | 1-034_validate_webhook_notifications/4-create-app | running command: [sh -c set -e cat << EOF | kubectl apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app-3 namespace: $NAMESPACE annotations: "notifications.argoproj.io/subscribe.test-on-created.test-webhook": "" spec: destination: namespace: $NAMESPACE server: https://kubernetes.default.svc project: default source: repoURL: https://github.com/redhat-developer/gitops-operator path: test/examples/nginx targetRevision: HEAD EOF ] logger.go:42: 07:00:30 | 1-034_validate_webhook_notifications/4-create-app | application.argoproj.io/my-app-3 created logger.go:42: 07:00:30 | 1-034_validate_webhook_notifications/4-create-app | running command: [sh -c sleep 10] === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 07:00:32 | 1-036_validate_keycloak_resource_reqs | Creating namespace: kuttl-test-guided-wombat logger.go:42: 07:00:32 | 1-036_validate_keycloak_resource_reqs/1-install | starting test step 1-install logger.go:42: 07:00:35 | 1-036_validate_keycloak_resource_reqs/1-install | ArgoCD:kuttl-test-guided-wombat/argocd created I0220 07:00:41.764335 122 request.go:682] Waited for 1.031981841s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1alpha5?timeout=32s === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 07:00:43 | 1-034_validate_webhook_notifications/4-create-app | test step completed 4-create-app logger.go:42: 07:00:43 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | starting test step 5-verify-webhook-notification logger.go:42: 07:00:43 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | running command: [sh -c #!/bin/bash kubectl -n $NAMESPACE logs deployment.apps/example-notifications-controller | grep "x509" PREV_CMD=$? if [ $PREV_CMD -eq 1 ]; then exit 0 else exit 1 fi ] logger.go:42: 07:00:43 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | error: error from server (NotFound): deployments.apps "example-notifications-controller" not found in namespace "kuttl-test-real-werewolf" logger.go:42: 07:00:43 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | running command: [sh -c #!/bin/bash # delays could occur in receiving notification, hence retry before failing for i in {1..5} do sleep 5 kubectl -n $NAMESPACE logs deployment.apps/webhook | grep '{"created":"my-app-3","type":"Directory"}' && break done ] logger.go:42: 07:00:49 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | [webhook] 2025/02/20 07:00:35 [57e7f3] executing /usr/bin/date (/usr/bin/date) with arguments ["/usr/bin/date"] and environment [PAYLOAD={"created":"my-app-3","type":"Directory"}] using /var/webhook as cwd logger.go:42: 07:00:51 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | test step completed 5-verify-webhook-notification logger.go:42: 07:00:51 | 1-034_validate_webhook_notifications | skipping kubernetes event logging logger.go:42: 07:00:51 | 1-034_validate_webhook_notifications | Deleting namespace: kuttl-test-real-werewolf === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 07:00:58 | 1-009_validate-manage-other-namespace | Ignoring README.md as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 07:00:58 | 1-009_validate-manage-other-namespace | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 07:00:58 | 1-009_validate-manage-other-namespace | Creating namespace: kuttl-test-still-mako logger.go:42: 07:00:58 | 1-009_validate-manage-other-namespace/1-install | starting test step 1-install I0220 07:00:59.231467 122 request.go:682] Waited for 1.029353882s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/metal3.io/v1alpha1?timeout=32s logger.go:42: 07:01:00 | 1-009_validate-manage-other-namespace/1-install | Namespace:/test-1-9-custom created logger.go:42: 07:01:00 | 1-009_validate-manage-other-namespace/1-install | ArgoCD:kuttl-test-still-mako/argocd created === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 07:01:07 | 1-036_validate_keycloak_resource_reqs/1-install | test step completed 1-install logger.go:42: 07:01:07 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | starting test step 2-update-sso-keycloak-provider logger.go:42: 07:01:07 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "add", "path": "/spec/sso", "value": {"provider": "keycloak"}}]' ] logger.go:42: 07:01:07 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | argocd.argoproj.io/argocd patched I0220 07:01:09.281016 122 request.go:682] Waited for 1.733206592s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 07:01:13 | 1-009_validate-manage-other-namespace/1-install | test step completed 1-install logger.go:42: 07:01:13 | 1-009_validate-manage-other-namespace/2-label-namespace | starting test step 2-label-namespace logger.go:42: 07:01:13 | 1-009_validate-manage-other-namespace/2-label-namespace | running command: [sh -c kubectl label ns test-1-9-custom argocd.argoproj.io/managed-by=$NAMESPACE] logger.go:42: 07:01:13 | 1-009_validate-manage-other-namespace/2-label-namespace | namespace/test-1-9-custom labeled logger.go:42: 07:01:16 | 1-009_validate-manage-other-namespace/2-label-namespace | test step completed 2-label-namespace logger.go:42: 07:01:16 | 1-009_validate-manage-other-namespace/3-check-secret | starting test step 3-check-secret logger.go:42: 07:01:16 | 1-009_validate-manage-other-namespace/3-check-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE,test-1-9-custom"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] I0220 07:01:19.294870 122 request.go:682] Waited for 2.23040085s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/coordination.k8s.io/v1?timeout=32s logger.go:42: 07:01:19 | 1-009_validate-manage-other-namespace/3-check-secret | test step completed 3-check-secret logger.go:42: 07:01:19 | 1-009_validate-manage-other-namespace/4-create-application | starting test step 4-create-application logger.go:42: 07:01:22 | 1-009_validate-manage-other-namespace/4-create-application | Application:kuttl-test-still-mako/test-1-9-custom created logger.go:42: 07:01:31 | 1-009_validate-manage-other-namespace/4-create-application | test step completed 4-create-application logger.go:42: 07:01:31 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | starting test step 5-unlabel-namespace logger.go:42: 07:01:31 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | running command: [sh -c kubectl label ns test-1-9-custom argocd.argoproj.io/managed-by-] logger.go:42: 07:01:31 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | namespace/test-1-9-custom unlabeled logger.go:42: 07:01:31 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | running command: [sh -c sleep 5] I0220 07:01:37.686706 122 request.go:682] Waited for 1.032872636s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/rbac.authorization.k8s.io/v1?timeout=32s logger.go:42: 07:01:39 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | test step completed 5-unlabel-namespace logger.go:42: 07:01:39 | 1-009_validate-manage-other-namespace/6-check-secret | starting test step 6-check-secret logger.go:42: 07:01:39 | 1-009_validate-manage-other-namespace/6-check-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] logger.go:42: 07:01:42 | 1-009_validate-manage-other-namespace/6-check-secret | test step completed 6-check-secret logger.go:42: 07:01:42 | 1-009_validate-manage-other-namespace/7-check | starting test step 7-check logger.go:42: 07:01:45 | 1-009_validate-manage-other-namespace/7-check | test step completed 7-check logger.go:42: 07:01:45 | 1-009_validate-manage-other-namespace/99-delete | starting test step 99-delete I0220 07:01:57.816066 122 request.go:682] Waited for 1.026305773s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/user.openshift.io/v1?timeout=32s logger.go:42: 07:01:59 | 1-009_validate-manage-other-namespace/99-delete | test step completed 99-delete logger.go:42: 07:01:59 | 1-009_validate-manage-other-namespace | skipping kubernetes event logging logger.go:42: 07:01:59 | 1-009_validate-manage-other-namespace | Deleting namespace: kuttl-test-still-mako === CONT kuttl/harness/1-005_validate_route_tls logger.go:42: 07:02:05 | 1-005_validate_route_tls | Creating namespace: kuttl-test-huge-man logger.go:42: 07:02:05 | 1-005_validate_route_tls/1-install | starting test step 1-install I0220 07:02:07.828671 122 request.go:682] Waited for 2.128651497s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 07:02:08 | 1-005_validate_route_tls/1-install | ArgoCD:kuttl-test-huge-man/example created === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 07:02:15 | 1-071_validate_SCC_HA/3-modify-argocd | test step completed 3-modify-argocd logger.go:42: 07:02:15 | 1-071_validate_SCC_HA/99-delete | starting test step 99-delete I0220 07:02:17.868837 122 request.go:682] Waited for 1.732655103s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1?timeout=32s logger.go:42: 07:02:18 | 1-071_validate_SCC_HA/99-delete | test step completed 99-delete logger.go:42: 07:02:18 | 1-071_validate_SCC_HA | skipping kubernetes event logging logger.go:42: 07:02:18 | 1-071_validate_SCC_HA | Deleting namespace: kuttl-test-just-turtle === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 07:02:59 | 1-030_validate_reencrypt | Creating namespace: kuttl-test-evolving-stud logger.go:42: 07:02:59 | 1-030_validate_reencrypt/1-install | starting test step 1-install I0220 07:03:00.880649 122 request.go:682] Waited for 1.032024396s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/security.internal.openshift.io/v1?timeout=32s logger.go:42: 07:03:02 | 1-030_validate_reencrypt/1-install | Namespace:/test-1-30-argo1 created logger.go:42: 07:03:02 | 1-030_validate_reencrypt/1-install | ArgoCD:test-1-30-argo1/argocd created === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 07:03:23 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | test step completed 2-update-sso-keycloak-provider logger.go:42: 07:03:23 | 1-036_validate_keycloak_resource_reqs/3-check-resources | starting test step 3-check-resources logger.go:42: 07:03:23 | 1-036_validate_keycloak_resource_reqs/3-check-resources | running command: [sh -c resources=$(oc get -n $NAMESPACE pod/keycloak-1-deploy -o jsonpath='{.spec.containers[0].resources}') if [[ -n ${resources} ]]; then if [[ "${resources}" != '{"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"250m","memory":"256Mi"}}' ]]; then echo "resources spec for pod/keycloak-1-deploy are incorrect" echo "${resources}" exit 1 fi else echo "resources spec not found in pod/keycloak-1-deploy" exit 1 fi ] I0220 07:03:24.663468 122 request.go:682] Waited for 1.031882712s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/build.openshift.io/v1?timeout=32s logger.go:42: 07:03:26 | 1-036_validate_keycloak_resource_reqs/3-check-resources | test step completed 3-check-resources logger.go:42: 07:03:26 | 1-036_validate_keycloak_resource_reqs | skipping kubernetes event logging logger.go:42: 07:03:26 | 1-036_validate_keycloak_resource_reqs | Deleting namespace: kuttl-test-guided-wombat === CONT kuttl/harness/1-005_validate_route_tls logger.go:42: 07:03:38 | 1-005_validate_route_tls/1-install | test step completed 1-install logger.go:42: 07:03:38 | 1-005_validate_route_tls/2-embedded-tls-data | starting test step 2-embedded-tls-data I0220 07:03:39.708705 122 request.go:682] Waited for 1.033549791s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 07:03:41 | 1-005_validate_route_tls/2-embedded-tls-data | ArgoCD:kuttl-test-huge-man/example updated logger.go:42: 07:03:42 | 1-005_validate_route_tls/2-embedded-tls-data | test step completed 2-embedded-tls-data logger.go:42: 07:03:42 | 1-005_validate_route_tls/3-secret-tls-data | starting test step 3-secret-tls-data === CONT kuttl/harness/1-007_validate_volume_mounts logger.go:42: 07:03:44 | 1-007_validate_volume_mounts | Creating namespace: kuttl-test-up-ocelot logger.go:42: 07:03:44 | 1-007_validate_volume_mounts/1-install | starting test step 1-install === CONT kuttl/harness/1-005_validate_route_tls logger.go:42: 07:03:44 | 1-005_validate_route_tls/3-secret-tls-data | ArgoCD:kuttl-test-huge-man/example updated logger.go:42: 07:03:45 | 1-005_validate_route_tls/3-secret-tls-data | Secret:kuttl-test-huge-man/tls-data-secret created === CONT kuttl/harness/1-007_validate_volume_mounts logger.go:42: 07:03:49 | 1-007_validate_volume_mounts/1-install | ArgoCD:kuttl-test-up-ocelot/argocd created === CONT kuttl/harness/1-005_validate_route_tls logger.go:42: 07:03:49 | 1-005_validate_route_tls/3-secret-tls-data | test step completed 3-secret-tls-data logger.go:42: 07:03:49 | 1-005_validate_route_tls/4-update-secret-tls-data | starting test step 4-update-secret-tls-data logger.go:42: 07:03:49 | 1-005_validate_route_tls/4-update-secret-tls-data | running command: [sh -c sleep 10] === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 07:03:49 | 1-030_validate_reencrypt/1-install | test step completed 1-install logger.go:42: 07:03:49 | 1-030_validate_reencrypt/2-wait | starting test step 2-wait logger.go:42: 07:03:49 | 1-030_validate_reencrypt/2-wait | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running "test-1-30-argo1" ] logger.go:42: 07:03:50 | 1-030_validate_reencrypt/2-wait | Waiting until all pods in namespace test-1-30-argo1 are up logger.go:42: 07:03:50 | 1-030_validate_reencrypt/2-wait | All pods are up: logger.go:42: 07:03:50 | 1-030_validate_reencrypt/2-wait | argocd-application-controller-0 1/1 Running 0 12s logger.go:42: 07:03:50 | 1-030_validate_reencrypt/2-wait | argocd-redis-755b9984bf-zqbhl 1/1 Running 0 12s logger.go:42: 07:03:50 | 1-030_validate_reencrypt/2-wait | argocd-repo-server-66697f94c6-nz2pm 1/1 Running 0 12s logger.go:42: 07:03:50 | 1-030_validate_reencrypt/2-wait | argocd-server-77644d6b4-ccf8h 1/1 Running 0 12s I0220 07:03:51.709859 122 request.go:682] Waited for 1.000217681s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/build.openshift.io/v1?timeout=32s logger.go:42: 07:03:53 | 1-030_validate_reencrypt/2-wait | test step completed 2-wait logger.go:42: 07:03:53 | 1-030_validate_reencrypt/3-check-route | starting test step 3-check-route logger.go:42: 07:03:53 | 1-030_validate_reencrypt/3-check-route | running command: [sh -c routeURL=$(oc -n test-1-30-argo1 get route argocd-server -o jsonpath='{.status.ingress[0].host}') if ! curl --silent -k https://${routeURL} | grep -q "Your browser does not support JavaScript."; then echo "Route not configured properly" exit 1 fi ] logger.go:42: 07:03:56 | 1-030_validate_reencrypt/3-check-route | test step completed 3-check-route logger.go:42: 07:03:56 | 1-030_validate_reencrypt | skipping kubernetes event logging logger.go:42: 07:03:56 | 1-030_validate_reencrypt | Deleting namespace: kuttl-test-evolving-stud I0220 07:04:01.748694 122 request.go:682] Waited for 2.29098873s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s === CONT kuttl/harness/1-005_validate_route_tls logger.go:42: 07:04:02 | 1-005_validate_route_tls/4-update-secret-tls-data | Secret:kuttl-test-huge-man/tls-data-secret updated === CONT kuttl/harness/1-007_validate_namespace_scoped_install logger.go:42: 07:04:02 | 1-007_validate_namespace_scoped_install | Creating namespace: kuttl-test-diverse-crow logger.go:42: 07:04:02 | 1-007_validate_namespace_scoped_install/1-install | starting test step 1-install === CONT kuttl/harness/1-005_validate_route_tls logger.go:42: 07:04:05 | 1-005_validate_route_tls/4-update-secret-tls-data | test step completed 4-update-secret-tls-data logger.go:42: 07:04:05 | 1-005_validate_route_tls | skipping kubernetes event logging === CONT kuttl/harness/1-007_validate_namespace_scoped_install logger.go:42: 07:04:06 | 1-007_validate_namespace_scoped_install/1-install | ArgoCD:kuttl-test-diverse-crow/argocd created === CONT kuttl/harness/1-005_validate_route_tls logger.go:42: 07:04:06 | 1-005_validate_route_tls | Deleting namespace: kuttl-test-huge-man === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 07:04:14 | 1-008_validate-custom-argocd-namespace | Creating namespace: kuttl-test-frank-glowworm logger.go:42: 07:04:14 | 1-008_validate-custom-argocd-namespace/1-install | starting test step 1-install I0220 07:04:16.093888 122 request.go:682] Waited for 1.039997041s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s logger.go:42: 07:04:17 | 1-008_validate-custom-argocd-namespace/1-install | Namespace:/test-1-8-custom created logger.go:42: 07:04:17 | 1-008_validate-custom-argocd-namespace/1-install | ArgoCD:test-1-8-custom/argocd created === CONT kuttl/harness/1-007_validate_volume_mounts logger.go:42: 07:04:21 | 1-007_validate_volume_mounts/1-install | test step completed 1-install logger.go:42: 07:04:21 | 1-007_validate_volume_mounts/2-install | starting test step 2-install logger.go:42: 07:04:24 | 1-007_validate_volume_mounts/2-install | ArgoCD:kuttl-test-up-ocelot/argocd updated logger.go:42: 07:04:27 | 1-007_validate_volume_mounts/2-install | test step completed 2-install logger.go:42: 07:04:27 | 1-007_validate_volume_mounts | skipping kubernetes event logging logger.go:42: 07:04:27 | 1-007_validate_volume_mounts | Deleting namespace: kuttl-test-up-ocelot === CONT kuttl/harness/1-021_validate_rolebindings logger.go:42: 07:04:34 | 1-021_validate_rolebindings | Creating namespace: kuttl-test-huge-hawk logger.go:42: 07:04:34 | 1-021_validate_rolebindings/1-install | starting test step 1-install I0220 07:04:35.710260 122 request.go:682] Waited for 1.028345211s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s logger.go:42: 07:04:37 | 1-021_validate_rolebindings/1-install | ArgoCD:kuttl-test-huge-hawk/argocd created === CONT kuttl/harness/1-007_validate_namespace_scoped_install logger.go:42: 07:04:38 | 1-007_validate_namespace_scoped_install/1-install | test step completed 1-install logger.go:42: 07:04:38 | 1-007_validate_namespace_scoped_install/2-check-cluster-secret | starting test step 2-check-cluster-secret logger.go:42: 07:04:38 | 1-007_validate_namespace_scoped_install/2-check-cluster-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] logger.go:42: 07:04:41 | 1-007_validate_namespace_scoped_install/2-check-cluster-secret | test step completed 2-check-cluster-secret logger.go:42: 07:04:41 | 1-007_validate_namespace_scoped_install | skipping kubernetes event logging logger.go:42: 07:04:41 | 1-007_validate_namespace_scoped_install | Deleting namespace: kuttl-test-diverse-crow === CONT kuttl/harness/1-019_validate_volume_mounts logger.go:42: 07:04:47 | 1-019_validate_volume_mounts | Creating namespace: kuttl-test-super-squirrel logger.go:42: 07:04:48 | 1-019_validate_volume_mounts/1-install | starting test step 1-install I0220 07:04:50.249499 122 request.go:682] Waited for 1.000268458s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 07:04:52 | 1-008_validate-custom-argocd-namespace/1-install | test step completed 1-install logger.go:42: 07:04:52 | 1-008_validate-custom-argocd-namespace/2-wait | starting test step 2-wait logger.go:42: 07:04:52 | 1-008_validate-custom-argocd-namespace/2-wait | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running "test-1-8-custom" ] === CONT kuttl/harness/1-019_validate_volume_mounts logger.go:42: 07:04:52 | 1-019_validate_volume_mounts/1-install | ArgoCD:kuttl-test-super-squirrel/argocd created === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 07:04:53 | 1-008_validate-custom-argocd-namespace/2-wait | Waiting until all pods in namespace test-1-8-custom are up logger.go:42: 07:04:53 | 1-008_validate-custom-argocd-namespace/2-wait | All pods are up: logger.go:42: 07:04:53 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-application-controller-0 1/1 Running 0 35s logger.go:42: 07:04:53 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-redis-755b9984bf-7msl7 1/1 Running 0 35s logger.go:42: 07:04:53 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-repo-server-b46d7c74-6prhm 1/1 Running 0 35s logger.go:42: 07:04:53 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-server-fc968c5c8-z5tlj 1/1 Running 0 35s logger.go:42: 07:04:56 | 1-008_validate-custom-argocd-namespace/2-wait | test step completed 2-wait logger.go:42: 07:04:56 | 1-008_validate-custom-argocd-namespace/3-create-app | starting test step 3-create-app logger.go:42: 07:05:00 | 1-008_validate-custom-argocd-namespace/3-create-app | Application:test-1-8-custom/validate-custom-argocd created logger.go:42: 07:05:08 | 1-008_validate-custom-argocd-namespace/3-create-app | test step completed 3-create-app logger.go:42: 07:05:08 | 1-008_validate-custom-argocd-namespace/99-delete | starting test step 99-delete === CONT kuttl/harness/1-021_validate_rolebindings logger.go:42: 07:05:08 | 1-021_validate_rolebindings/1-install | test step completed 1-install logger.go:42: 07:05:08 | 1-021_validate_rolebindings | skipping kubernetes event logging logger.go:42: 07:05:09 | 1-021_validate_rolebindings | Deleting namespace: kuttl-test-huge-hawk === CONT kuttl/harness/1-029_validate_tls_secret_no_scale logger.go:42: 07:05:15 | 1-029_validate_tls_secret_no_scale | Creating namespace: kuttl-test-working-eel logger.go:42: 07:05:17 | 1-029_validate_tls_secret_no_scale/1-install | starting test step 1-install I0220 07:05:19.668798 122 request.go:682] Waited for 1.000674894s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 07:05:21 | 1-029_validate_tls_secret_no_scale/1-install | ArgoCD:kuttl-test-working-eel/argocd created === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 07:05:23 | 1-008_validate-custom-argocd-namespace/99-delete | test step completed 99-delete logger.go:42: 07:05:23 | 1-008_validate-custom-argocd-namespace | skipping kubernetes event logging logger.go:42: 07:05:23 | 1-008_validate-custom-argocd-namespace | Deleting namespace: kuttl-test-frank-glowworm === CONT kuttl/harness/1-019_validate_volume_mounts logger.go:42: 07:05:24 | 1-019_validate_volume_mounts/1-install | test step completed 1-install logger.go:42: 07:05:24 | 1-019_validate_volume_mounts | skipping kubernetes event logging logger.go:42: 07:05:24 | 1-019_validate_volume_mounts | Deleting namespace: kuttl-test-super-squirrel === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 07:05:29 | 1-023_validate_repo_server_tls | Creating namespace: kuttl-test-improved-spider logger.go:42: 07:05:29 | 1-023_validate_repo_server_tls/1-install | starting test step 1-install I0220 07:05:30.687557 122 request.go:682] Waited for 1.031527315s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta2?timeout=32s === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:05:32 | 1-025-validate-managed-by-change | Creating namespace: kuttl-test-hip-yeti logger.go:42: 07:05:32 | 1-025-validate-managed-by-change/1-install | starting test step 1-install === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 07:05:32 | 1-023_validate_repo_server_tls/1-install | Namespace:/test-1-23-custom created logger.go:42: 07:05:32 | 1-023_validate_repo_server_tls/1-install | ArgoCD:test-1-23-custom/argocd created === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:05:34 | 1-025-validate-managed-by-change/1-install | Namespace:/test-1-25-target created logger.go:42: 07:05:34 | 1-025-validate-managed-by-change/1-install | Namespace:/test-1-25-argo1 created logger.go:42: 07:05:34 | 1-025-validate-managed-by-change/1-install | Namespace:/test-1-25-argo2 created logger.go:42: 07:05:34 | 1-025-validate-managed-by-change/1-install | ArgoCD:test-1-25-argo1/argocd created logger.go:42: 07:05:34 | 1-025-validate-managed-by-change/1-install | ArgoCD:test-1-25-argo2/argocd created === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 07:05:44 | 1-023_validate_repo_server_tls/1-install | test step completed 1-install logger.go:42: 07:05:44 | 1-023_validate_repo_server_tls/2-change-argocd | starting test step 2-change-argocd I0220 07:05:46.011183 122 request.go:682] Waited for 1.032377004s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 07:05:47 | 1-023_validate_repo_server_tls/2-change-argocd | ArgoCD:test-1-23-custom/argocd updated logger.go:42: 07:05:48 | 1-023_validate_repo_server_tls/2-change-argocd | test step completed 2-change-argocd logger.go:42: 07:05:48 | 1-023_validate_repo_server_tls/4-install-app | starting test step 4-install-app logger.go:42: 07:05:51 | 1-023_validate_repo_server_tls/4-install-app | Application:test-1-23-custom/guestbook created === CONT kuttl/harness/1-029_validate_tls_secret_no_scale logger.go:42: 07:05:53 | 1-029_validate_tls_secret_no_scale/1-install | test step completed 1-install logger.go:42: 07:05:53 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | starting test step 2-create-tls-secret logger.go:42: 07:05:53 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | running command: [sh -c openssl req -x509 -newkey rsa:4096 -keyout /tmp/test-029-key.pem -out /tmp/test-029-cert.pem -subj '/CN=test-029-cert' -days 365 -nodes cert=$(cat /tmp/test-029-cert.pem | base64 -w 0) key=$(cat /tmp/test-029-key.pem | base64 -w 0) # Dirty hack to replace argocd-tls cat <<_EOF_ | oc replace -n $NAMESPACE secret argocd-tls -f - apiVersion: v1 kind: Secret type: kubernetes.io/tls metadata: name: argocd-tls namespace: $NAMESPACE data: tls.key: $key tls.crt: $cert _EOF_ ] logger.go:42: 07:05:53 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | ......+.+...+++++++++++++++++++++++++++++++++++++++++++++*...+..........+........+......+.......+.........+.....+......+.........+..........+++++++++++++++++++++++++++++++++++++++++++++*..+....+......+.....+.+..............+.............+.................+..........+...+........+...+.+........+......+......+...+......+.+..................+........+.......+.................+....+..+....+...+......+.....+.......+...+.........+..+....+.....+++++ logger.go:42: 07:05:53 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | ..+...+...+..+......+...............+......+....+.........+.....+......+++++++++++++++++++++++++++++++++++++++++++++*.....+.+...+..+....+.....+.......+........+.......+........+.......+.....+...+.........+.+.........+++++++++++++++++++++++++++++++++++++++++++++*.+........+.......+...+..+++++ logger.go:42: 07:05:53 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | ----- logger.go:42: 07:05:54 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | secret/argocd-tls replaced I0220 07:05:56.023541 122 request.go:682] Waited for 1.450654924s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/controlplane.operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 07:05:57 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | test step completed 2-create-tls-secret logger.go:42: 07:05:57 | 1-029_validate_tls_secret_no_scale/3-sleep-and-recheck | starting test step 3-sleep-and-recheck logger.go:42: 07:05:57 | 1-029_validate_tls_secret_no_scale/3-sleep-and-recheck | running command: [sh -c sleep 10] === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 07:05:59 | 1-023_validate_repo_server_tls/4-install-app | test step completed 4-install-app logger.go:42: 07:05:59 | 1-023_validate_repo_server_tls/99-delete | starting test step 99-delete === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:06:07 | 1-025-validate-managed-by-change/1-install | test step completed 1-install logger.go:42: 07:06:07 | 1-025-validate-managed-by-change/2-install-app | starting test step 2-install-app logger.go:42: 07:06:07 | 1-025-validate-managed-by-change/2-install-app | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running "test-1-25-argo1" wait_until_pods_running "test-1-25-argo2" ] logger.go:42: 07:06:08 | 1-025-validate-managed-by-change/2-install-app | Waiting until all pods in namespace test-1-25-argo1 are up logger.go:42: 07:06:08 | 1-025-validate-managed-by-change/2-install-app | All pods are up: logger.go:42: 07:06:08 | 1-025-validate-managed-by-change/2-install-app | argocd-application-controller-0 1/1 Running 0 33s logger.go:42: 07:06:08 | 1-025-validate-managed-by-change/2-install-app | argocd-redis-755b9984bf-kgtb8 1/1 Running 0 33s logger.go:42: 07:06:08 | 1-025-validate-managed-by-change/2-install-app | argocd-repo-server-6f4d9859b6-c5856 1/1 Running 0 33s logger.go:42: 07:06:08 | 1-025-validate-managed-by-change/2-install-app | argocd-server-bb7969866-ln6k8 1/1 Running 0 33s I0220 07:06:09.882531 122 request.go:682] Waited for 1.000296079s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/admissionregistration.k8s.io/v1?timeout=32s logger.go:42: 07:06:09 | 1-025-validate-managed-by-change/2-install-app | Waiting until all pods in namespace test-1-25-argo2 are up logger.go:42: 07:06:09 | 1-025-validate-managed-by-change/2-install-app | All pods are up: logger.go:42: 07:06:09 | 1-025-validate-managed-by-change/2-install-app | argocd-application-controller-0 1/1 Running 0 33s logger.go:42: 07:06:09 | 1-025-validate-managed-by-change/2-install-app | argocd-redis-755b9984bf-7q7gh 1/1 Running 0 33s logger.go:42: 07:06:09 | 1-025-validate-managed-by-change/2-install-app | argocd-repo-server-cb4656d45-vwthm 1/1 Running 0 33s logger.go:42: 07:06:09 | 1-025-validate-managed-by-change/2-install-app | argocd-server-64848c4f4c-dpt5h 1/1 Running 0 33s === CONT kuttl/harness/1-029_validate_tls_secret_no_scale logger.go:42: 07:06:12 | 1-029_validate_tls_secret_no_scale/3-sleep-and-recheck | test step completed 3-sleep-and-recheck logger.go:42: 07:06:12 | 1-029_validate_tls_secret_no_scale | skipping kubernetes event logging logger.go:42: 07:06:19 | 1-029_validate_tls_secret_no_scale | Deleting namespace: kuttl-test-working-eel === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:06:22 | 1-025-validate-managed-by-change/2-install-app | Application:test-1-25-argo1/guestbook created I0220 07:06:22.833790 122 request.go:682] Waited for 1.024660054s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/argoproj.io/v1beta1?timeout=32s === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:06:30 | 1-012_validate-managed-by-chain | Ignoring README.md as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 07:06:30 | 1-023_validate_repo_server_tls/99-delete | test step completed 99-delete logger.go:42: 07:06:30 | 1-023_validate_repo_server_tls | skipping kubernetes event logging === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:06:30 | 1-012_validate-managed-by-chain | Creating namespace: kuttl-test-genuine-feline logger.go:42: 07:06:30 | 1-012_validate-managed-by-chain/1-install | starting test step 1-install === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 07:06:33 | 1-023_validate_repo_server_tls | Deleting namespace: kuttl-test-improved-spider I0220 07:06:34.784631 122 request.go:682] Waited for 1.050093518s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/security.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:06:37 | 1-025-validate-managed-by-change/2-install-app | test step completed 2-install-app logger.go:42: 07:06:37 | 1-025-validate-managed-by-change/3-update-namespace | starting test step 3-update-namespace === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:06:41 | 1-012_validate-managed-by-chain/1-install | Namespace:/test-1-12-custom created === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 07:06:44 | 1-003_alpha_to_beta_sso_conflict_conversion | Creating namespace: kuttl-test-faithful-albacore === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:06:44 | 1-012_validate-managed-by-chain/1-install | Namespace:/test-1-12-custom2 created === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 07:06:45 | 1-003_alpha_to_beta_sso_conflict_conversion/1-argocd-dex-keycloak-conflict | starting test step 1-argocd-dex-keycloak-conflict === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:06:46 | 1-012_validate-managed-by-chain/1-install | ArgoCD:kuttl-test-genuine-feline/argocd created === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:06:46 | 1-025-validate-managed-by-change/3-update-namespace | Namespace:/test-1-25-target updated I0220 07:06:47.520862 122 request.go:682] Waited for 1.05010492s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/image.openshift.io/v1?timeout=32s logger.go:42: 07:06:50 | 1-025-validate-managed-by-change/3-update-namespace | Application:test-1-25-argo1/guestbook updated logger.go:42: 07:06:53 | 1-025-validate-managed-by-change/3-update-namespace | Application:test-1-25-argo2/guestbook created === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 07:06:55 | 1-003_alpha_to_beta_sso_conflict_conversion/1-argocd-dex-keycloak-conflict | ArgoCD:kuttl-test-faithful-albacore/argocd created === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:06:57 | 1-012_validate-managed-by-chain/1-install | test step completed 1-install logger.go:42: 07:06:57 | 1-012_validate-managed-by-chain/2-label-namespace | starting test step 2-label-namespace logger.go:42: 07:06:57 | 1-012_validate-managed-by-chain/2-label-namespace | running command: [sh -c kubectl label ns test-1-12-custom argocd.argoproj.io/managed-by=$NAMESPACE --overwrite] === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:07:00 | 1-025-validate-managed-by-change/3-update-namespace | test step completed 3-update-namespace logger.go:42: 07:07:00 | 1-025-validate-managed-by-change/99-delete | starting test step 99-delete === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:07:05 | 1-012_validate-managed-by-chain/2-label-namespace | namespace/test-1-12-custom labeled I0220 07:07:06.309808 122 request.go:682] Waited for 1.000351966s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/oauth.openshift.io/v1?timeout=32s logger.go:42: 07:07:11 | 1-012_validate-managed-by-chain/2-label-namespace | test step completed 2-label-namespace logger.go:42: 07:07:11 | 1-012_validate-managed-by-chain/3-check-secret | starting test step 3-check-secret logger.go:42: 07:07:11 | 1-012_validate-managed-by-chain/3-check-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE,test-1-12-custom"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] logger.go:42: 07:07:14 | 1-012_validate-managed-by-chain/3-check-secret | test step completed 3-check-secret logger.go:42: 07:07:14 | 1-012_validate-managed-by-chain/4-create-application | starting test step 4-create-application I0220 07:07:16.314256 122 request.go:682] Waited for 1.200737039s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/security.internal.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:07:21 | 1-025-validate-managed-by-change/99-delete | test step completed 99-delete logger.go:42: 07:07:21 | 1-025-validate-managed-by-change | skipping kubernetes event logging === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 07:07:26 | 1-003_alpha_to_beta_sso_conflict_conversion/1-argocd-dex-keycloak-conflict | test step completed 1-argocd-dex-keycloak-conflict logger.go:42: 07:07:26 | 1-003_alpha_to_beta_sso_conflict_conversion/2-delete | starting test step 2-delete === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:07:26 | 1-012_validate-managed-by-chain/4-create-application | Application:kuttl-test-genuine-feline/test-1-12-custom created logger.go:42: 07:07:39 | 1-012_validate-managed-by-chain/4-create-application | test step completed 4-create-application logger.go:42: 07:07:39 | 1-012_validate-managed-by-chain/7-label-namespace | starting test step 7-label-namespace logger.go:42: 07:07:39 | 1-012_validate-managed-by-chain/7-label-namespace | running command: [sh -c kubectl label ns test-1-12-custom2 argocd.argoproj.io/managed-by=$NAMESPACE --overwrite] logger.go:42: 07:07:39 | 1-012_validate-managed-by-chain/7-label-namespace | namespace/test-1-12-custom2 labeled === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 07:07:39 | 1-025-validate-managed-by-change | Deleting namespace: kuttl-test-hip-yeti I0220 07:07:42.285320 122 request.go:682] Waited for 1.049984581s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/admissionregistration.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-004_beta_to_alpha_conversion logger.go:42: 07:07:49 | 1-004_beta_to_alpha_conversion | Creating namespace: kuttl-test-casual-sole === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 07:07:49 | 1-003_alpha_to_beta_sso_conflict_conversion/2-delete | test step completed 2-delete logger.go:42: 07:07:49 | 1-003_alpha_to_beta_sso_conflict_conversion | skipping kubernetes event logging === CONT kuttl/harness/1-004_beta_to_alpha_conversion logger.go:42: 07:07:49 | 1-004_beta_to_alpha_conversion/1-argocd-dex | starting test step 1-argocd-dex === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:07:49 | 1-012_validate-managed-by-chain/7-label-namespace | test step completed 7-label-namespace logger.go:42: 07:07:49 | 1-012_validate-managed-by-chain/8-create-application | starting test step 8-create-application === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 07:07:49 | 1-003_alpha_to_beta_sso_conflict_conversion | Deleting namespace: kuttl-test-faithful-albacore === CONT kuttl/harness/1-004_beta_to_alpha_conversion logger.go:42: 07:07:55 | 1-004_beta_to_alpha_conversion/1-argocd-dex | ArgoCD:kuttl-test-casual-sole/argocd created === CONT kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha I0220 07:07:56.086980 122 request.go:682] Waited for 1.041325072s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta3?timeout=32s logger.go:42: 07:07:57 | 1-067_validate_redis_secure_comm_no_autotls_ha | Creating namespace: kuttl-test-quick-foal logger.go:42: 07:07:57 | 1-067_validate_redis_secure_comm_no_autotls_ha/1-install | starting test step 1-install === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:07:57 | 1-012_validate-managed-by-chain/8-create-application | Application:kuttl-test-genuine-feline/test-1-12-custom2 created === CONT kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha logger.go:42: 07:08:00 | 1-067_validate_redis_secure_comm_no_autotls_ha/1-install | ArgoCD:kuttl-test-quick-foal/argocd created === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:08:02 | 1-012_validate-managed-by-chain/8-create-application | test step completed 8-create-application logger.go:42: 07:08:02 | 1-012_validate-managed-by-chain/9-delete-app-and-namespace | starting test step 9-delete-app-and-namespace I0220 07:08:14.752084 122 request.go:682] Waited for 1.032868004s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/pipelines.openshift.io/v1alpha1?timeout=32s logger.go:42: 07:08:16 | 1-012_validate-managed-by-chain/9-delete-app-and-namespace | test step completed 9-delete-app-and-namespace logger.go:42: 07:08:16 | 1-012_validate-managed-by-chain/10-check-secret | starting test step 10-check-secret logger.go:42: 07:08:16 | 1-012_validate-managed-by-chain/10-check-secret | running command: [sh -c should="$NAMESPACE,test-1-12-custom2" namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$should"; then echo "Assertion for cluster secret failed! '$namespaces' != '$should'" # 1.2.0 doesn't reconcile the cluster secret on namespace deletion if test GITOPS_TARGET_VERSION = "1.2.0"; then exit 0 else exit 1 fi fi exit 0 ] logger.go:42: 07:08:19 | 1-012_validate-managed-by-chain/10-check-secret | test step completed 10-check-secret logger.go:42: 07:08:19 | 1-012_validate-managed-by-chain/11-recreate-applications | starting test step 11-recreate-applications logger.go:42: 07:08:21 | 1-012_validate-managed-by-chain/11-recreate-applications | Namespace:/test-1-12-custom created logger.go:42: 07:08:21 | 1-012_validate-managed-by-chain/11-recreate-applications | Application:kuttl-test-genuine-feline/test-1-12-custom created logger.go:42: 07:08:21 | 1-012_validate-managed-by-chain/11-recreate-applications | Application:kuttl-test-genuine-feline/test-1-12-custom2 created logger.go:42: 07:08:22 | 1-012_validate-managed-by-chain/11-recreate-applications | test step completed 11-recreate-applications logger.go:42: 07:08:22 | 1-012_validate-managed-by-chain/99-delete | starting test step 99-delete === CONT kuttl/harness/1-004_beta_to_alpha_conversion logger.go:42: 07:08:26 | 1-004_beta_to_alpha_conversion/1-argocd-dex | test step completed 1-argocd-dex logger.go:42: 07:08:26 | 1-004_beta_to_alpha_conversion/2-delete | starting test step 2-delete I0220 07:08:28.030709 122 request.go:682] Waited for 1.03333001s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/node.k8s.io/v1?timeout=32s logger.go:42: 07:08:29 | 1-004_beta_to_alpha_conversion/2-delete | test step completed 2-delete logger.go:42: 07:08:29 | 1-004_beta_to_alpha_conversion | skipping kubernetes event logging logger.go:42: 07:08:29 | 1-004_beta_to_alpha_conversion | Deleting namespace: kuttl-test-casual-sole === CONT kuttl/harness/1-002_alpha_to_beta_keycloak_conversion === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:08:36 | 1-012_validate-managed-by-chain/99-delete | test step completed 99-delete logger.go:42: 07:08:36 | 1-012_validate-managed-by-chain | skipping kubernetes event logging === CONT kuttl/harness/1-002_alpha_to_beta_keycloak_conversion logger.go:42: 07:08:36 | 1-002_alpha_to_beta_keycloak_conversion | Creating namespace: kuttl-test-free-emu logger.go:42: 07:08:37 | 1-002_alpha_to_beta_keycloak_conversion/1-argocd-keycloak | starting test step 1-argocd-keycloak === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 07:08:37 | 1-012_validate-managed-by-chain | Deleting namespace: kuttl-test-genuine-feline I0220 07:08:38.064516 122 request.go:682] Waited for 1.031759809s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/metal3.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-002_alpha_to_beta_keycloak_conversion logger.go:42: 07:08:39 | 1-002_alpha_to_beta_keycloak_conversion/1-argocd-keycloak | ArgoCD:kuttl-test-free-emu/argocd created === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 07:08:42 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | test step completed 2-enable_autotls logger.go:42: 07:08:42 | 1-069_validate_redis_secure_comm_autotls_ha/3-check_secret | starting test step 3-check_secret logger.go:42: 07:08:42 | 1-069_validate_redis_secure_comm_autotls_ha/3-check_secret | running command: [sh -c set -e secret_type="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{.type}}')" secret_len="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{len .data}}')" expected_secret_type="kubernetes.io/tls" expected_secret_len=2 if test ${secret_type} != ${expected_secret_type}; then echo "argocd-operator-redis-tls secret type is ${secret_type} and should be ${expected_secret_type}" exit 1 fi if test ${secret_len} != ${expected_secret_len}; then echo "argocd-operator-redis-tls secret length is ${secret_len} and should be ${expected_secret_len}" exit 1 fi ] === CONT kuttl/harness/1-003_validate_console_link logger.go:42: 07:08:43 | 1-003_validate_console_link | Creating namespace: kuttl-test-loving-whale logger.go:42: 07:08:43 | 1-003_validate_console_link/1-check-console-link | starting test step 1-check-console-link logger.go:42: 07:08:45 | 1-003_validate_console_link/1-check-console-link | test step completed 1-check-console-link logger.go:42: 07:08:45 | 1-003_validate_console_link | skipping kubernetes event logging logger.go:42: 07:08:45 | 1-003_validate_console_link | Deleting namespace: kuttl-test-loving-whale I0220 07:08:48.073212 122 request.go:682] Waited for 2.131829788s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/build.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 07:08:48 | 1-069_validate_redis_secure_comm_autotls_ha/3-check_secret | test step completed 3-check_secret logger.go:42: 07:08:48 | 1-069_validate_redis_secure_comm_autotls_ha | skipping kubernetes event logging logger.go:42: 07:08:48 | 1-069_validate_redis_secure_comm_autotls_ha | Deleting namespace: kuttl-test-major-blowfish === CONT kuttl/harness/1-002_alpha_to_beta_keycloak_conversion logger.go:42: 07:09:11 | 1-002_alpha_to_beta_keycloak_conversion/1-argocd-keycloak | test step completed 1-argocd-keycloak logger.go:42: 07:09:11 | 1-002_alpha_to_beta_keycloak_conversion/2-delete | starting test step 2-delete I0220 07:09:13.092383 122 request.go:682] Waited for 1.031242111s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1beta1?timeout=32s logger.go:42: 07:09:14 | 1-002_alpha_to_beta_keycloak_conversion/2-delete | test step completed 2-delete logger.go:42: 07:09:14 | 1-002_alpha_to_beta_keycloak_conversion | skipping kubernetes event logging logger.go:42: 07:09:14 | 1-002_alpha_to_beta_keycloak_conversion | Deleting namespace: kuttl-test-free-emu === CONT kuttl/harness/1-042_restricted_pss_compliant logger.go:42: 07:10:50 | 1-042_restricted_pss_compliant/1-install-argocd-in-restricted-pss-ns copy | test step failed 1-install-argocd-in-restricted-pss-ns copy case.go:364: failed in step 1-install-argocd-in-restricted-pss-ns copy case.go:366: --- Namespace:/test-1-042-restricted-pss-compliant +++ Namespace:/test-1-042-restricted-pss-compliant @@ -1,12 +1,72 @@ apiVersion: v1 kind: Namespace metadata: + annotations: + openshift.io/sa.scc.mcs: s0:c40,c5 + openshift.io/sa.scc.supplemental-groups: 1001570000/10000 + openshift.io/sa.scc.uid-range: 1001570000/10000 labels: + kubernetes.io/metadata.name: test-1-042-restricted-pss-compliant + olm.operatorgroup.uid/4f5a97c4-56ff-4009-ae21-94a919a61f2a: "" + openshift.io/cluster-monitoring: "true" pod-security.kubernetes.io/audit: restricted - pod-security.kubernetes.io/audit-version: latest + pod-security.kubernetes.io/audit-version: v1.24 pod-security.kubernetes.io/enforce: restricted pod-security.kubernetes.io/enforce-version: latest pod-security.kubernetes.io/warn: restricted - pod-security.kubernetes.io/warn-version: latest + pod-security.kubernetes.io/warn-version: v1.24 + managedFields: + - apiVersion: v1 + fieldsType: FieldsV1 + fieldsV1: + f:metadata: + f:labels: + .: {} + f:kubernetes.io/metadata.name: {} + f:pod-security.kubernetes.io/enforce: {} + f:pod-security.kubernetes.io/enforce-version: {} + manager: kubectl-kuttl + operation: Update + time: "2025-02-20T06:58:49Z" + - apiVersion: v1 + fieldsType: FieldsV1 + fieldsV1: + f:metadata: + f:labels: + f:openshift.io/cluster-monitoring: {} + manager: manager + operation: Update + time: "2025-02-20T06:58:49Z" + - apiVersion: v1 + fieldsType: FieldsV1 + fieldsV1: + f:metadata: + f:labels: + f:olm.operatorgroup.uid/4f5a97c4-56ff-4009-ae21-94a919a61f2a: {} + manager: olm + operation: Update + time: "2025-02-20T06:59:20Z" + - apiVersion: v1 + fieldsType: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: {} + f:openshift.io/sa.scc.mcs: {} + f:openshift.io/sa.scc.supplemental-groups: {} + f:openshift.io/sa.scc.uid-range: {} + f:labels: + f:pod-security.kubernetes.io/audit: {} + f:pod-security.kubernetes.io/audit-version: {} + f:pod-security.kubernetes.io/warn: {} + f:pod-security.kubernetes.io/warn-version: {} + manager: cluster-policy-controller + operation: Update + time: "2025-02-20T07:03:37Z" name: test-1-042-restricted-pss-compliant +spec: + finalizers: + - kubernetes +status: + phase: Active case.go:366: resource Namespace:/test-1-042-restricted-pss-compliant: .metadata.labels.pod-security.kubernetes.io/warn-version: value mismatch, expected: latest != actual: v1.24 logger.go:42: 07:10:50 | 1-042_restricted_pss_compliant | skipping kubernetes event logging logger.go:42: 07:10:50 | 1-042_restricted_pss_compliant | Deleting namespace: kuttl-test-dear-mako === CONT kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha logger.go:42: 07:17:03 | 1-067_validate_redis_secure_comm_no_autotls_ha/1-install | test step completed 1-install logger.go:42: 07:17:03 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | starting test step 2-generate_cert logger.go:42: 07:17:03 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | running command: [sh -c set -e if [[ $(uname -o) == "Darwin" ]]; then echo "\n[SAN]\nsubjectAltName=DNS:argocd-redis.$NAMESPACE.svc.cluster.local\n[req]\ndistinguished_name=req" > ${PWD}/openssl_test.cnf else echo -e "\n[SAN]\nsubjectAltName=DNS:argocd-redis.$NAMESPACE.svc.cluster.local\n[req]\ndistinguished_name=req" > ${PWD}/openssl_test.cnf fi openssl req -new -x509 \ -subj "/C=XX/ST=XX/O=Testing/CN=redis" \ -reqexts SAN -extensions SAN \ -config ${PWD}/openssl_test.cnf \ -keyout ${PWD}/redis-ha.key \ -out ${PWD}/redis-ha.crt \ -newkey rsa:4096 \ -nodes \ -sha256 \ -days 10 ] logger.go:42: 07:17:03 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | ..+++++++++++++++++++++++++++++++++++++++++++++*.....+......+.+...+..+...+......+....+..+....+.........+++++++++++++++++++++++++++++++++++++++++++++*......+..+....+...........+.+..+.......+.....+...+..........+...................................+.............+...+.........+..+.+...+............+...+..+....+.....+...............+....+.................+....+.....+....+++++ logger.go:42: 07:17:03 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | .................+.....+.+.....+....+......+......+...+...+........+.........+...+...+.+..............+++++++++++++++++++++++++++++++++++++++++++++*.+.+............+..+..........+...+........+......+.+........+......+.+........+.......+...+..+...+................+...+..+...+...+....+......+..+.+......+........+++++++++++++++++++++++++++++++++++++++++++++*...............+........+............+.+..+......+.....................+.+...+.........+..+...+...............+.+.....+.+.........+...........+....+..+.......+.....................+.........+...+.........+...........+....+........+...+....+...+.................+....+..+...+....+.....+.......+....................+.+..+..........+........+............+...+...+..........+.....+......+.+.....+............+.......+...+...+........+..........+......+.....+....+...............+..............+............+++++ logger.go:42: 07:17:03 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | ----- I0220 07:17:05.127470 122 request.go:682] Waited for 1.000016594s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 07:17:07 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | test step completed 2-generate_cert logger.go:42: 07:17:07 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | starting test step 3-create_secret logger.go:42: 07:17:07 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | running command: [sh -c set -e # clean up key and cert and config file cleanup() { rm -rf ${PWD}/redis-ha.crt && rm -rf ${PWD}/redis-ha.key && rm -rf ${PWD}/openssl_test.cnf } trap cleanup INT TERM EXIT oc create secret tls argocd-operator-redis-tls --key=${PWD}/redis-ha.key --cert=${PWD}/redis-ha.crt -n $NAMESPACE ] logger.go:42: 07:17:07 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | secret/argocd-operator-redis-tls created logger.go:42: 07:17:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | test step completed 3-create_secret logger.go:42: 07:17:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | starting test step 4-create_annotation logger.go:42: 07:17:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | running command: [sh -c set -e oc annotate secret argocd-operator-redis-tls argocds.argoproj.io/name=argocd -n $NAMESPACE ] logger.go:42: 07:17:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | secret/argocd-operator-redis-tls annotated I0220 07:17:18.734884 122 request.go:682] Waited for 1.04028901s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 07:26:22 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | test step completed 4-create_annotation logger.go:42: 07:26:22 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | starting test step 5-check_redis_config logger.go:42: 07:26:22 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | running command: [sh -c set -e # Define the expected configurations expected_redis_config=( 'port 0' 'tls-port 6379' 'tls-cert-file /app/config/redis/tls/tls.crt' 'tls-ca-cert-file /app/config/redis/tls/tls.crt' 'tls-key-file /app/config/redis/tls/tls.key' 'tls-replication yes' 'tls-auth-clients no' ) expected_sentinel_config=( 'port 0' 'tls-port 26379' 'tls-cert-file "/app/config/redis/tls/tls.crt"' 'tls-ca-cert-file "/app/config/redis/tls/tls.crt"' 'tls-key-file "/app/config/redis/tls/tls.key"' 'tls-replication yes' 'tls-auth-clients no' ) # Function to execute and retry the command if necessary execute_with_retry() { local config=$1 local file=$2 local attempts=3 local previous_time=0 local found=false for attempt in $(seq 1 $attempts); do # Calculate the time interval (increasing with each attempt) interval=$((attempt + previous_time)) echo "Attempt $attempt for '${config}' (Waiting for ${interval}s before execution)" # Execute the command and check if the expected result is found if oc exec -i pod/argocd-redis-ha-server-0 -n $NAMESPACE -c redis -- grep "${config}" $file > /dev/null 2>&1; then echo "Found expected config '${config}' on attempt $attempt." found=true break else echo "Expected config '${config}' not found on attempt $attempt. Retrying..." sleep $interval fi # Update previous_time for the next attempt previous_time=$interval done # If the config was not found after all attempts, log the failure if [ "$found" = false ]; then echo "Failed to find expected config '${config}' after $attempts attempts." fi } # Loop through the redis config and check with retry logic for config in "${expected_redis_config[@]}" do execute_with_retry "$config" "/data/conf/redis.conf" done # Loop through the sentinel config and check with retry logic for config in "${expected_sentinel_config[@]}" do execute_with_retry "$config" "/data/conf/sentinel.conf" done ] logger.go:42: 07:26:22 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'port 0' (Waiting for 1s before execution) logger.go:42: 07:26:22 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'port 0' on attempt 1. logger.go:42: 07:26:22 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-port 6379' (Waiting for 1s before execution) logger.go:42: 07:26:23 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-port 6379' on attempt 1. logger.go:42: 07:26:23 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-cert-file /app/config/redis/tls/tls.crt' (Waiting for 1s before execution) logger.go:42: 07:26:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-cert-file /app/config/redis/tls/tls.crt' on attempt 1. logger.go:42: 07:26:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-ca-cert-file /app/config/redis/tls/tls.crt' (Waiting for 1s before execution) logger.go:42: 07:26:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-ca-cert-file /app/config/redis/tls/tls.crt' on attempt 1. logger.go:42: 07:26:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-key-file /app/config/redis/tls/tls.key' (Waiting for 1s before execution) logger.go:42: 07:26:25 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-key-file /app/config/redis/tls/tls.key' on attempt 1. logger.go:42: 07:26:25 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-replication yes' (Waiting for 1s before execution) logger.go:42: 07:26:25 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-replication yes' on attempt 1. logger.go:42: 07:26:25 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-auth-clients no' (Waiting for 1s before execution) logger.go:42: 07:26:26 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-auth-clients no' on attempt 1. logger.go:42: 07:26:26 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'port 0' (Waiting for 1s before execution) logger.go:42: 07:26:26 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'port 0' on attempt 1. logger.go:42: 07:26:26 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-port 26379' (Waiting for 1s before execution) logger.go:42: 07:26:27 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-port 26379' on attempt 1. logger.go:42: 07:26:27 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-cert-file "/app/config/redis/tls/tls.crt"' (Waiting for 1s before execution) logger.go:42: 07:26:27 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-cert-file "/app/config/redis/tls/tls.crt"' on attempt 1. logger.go:42: 07:26:27 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-ca-cert-file "/app/config/redis/tls/tls.crt"' (Waiting for 1s before execution) logger.go:42: 07:26:28 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-ca-cert-file "/app/config/redis/tls/tls.crt"' on attempt 1. logger.go:42: 07:26:28 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-key-file "/app/config/redis/tls/tls.key"' (Waiting for 1s before execution) logger.go:42: 07:26:28 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-key-file "/app/config/redis/tls/tls.key"' on attempt 1. logger.go:42: 07:26:28 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-replication yes' (Waiting for 1s before execution) logger.go:42: 07:26:29 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-replication yes' on attempt 1. logger.go:42: 07:26:29 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Attempt 1 for 'tls-auth-clients no' (Waiting for 1s before execution) logger.go:42: 07:26:29 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | Found expected config 'tls-auth-clients no' on attempt 1. I0220 07:26:30.857860 122 request.go:682] Waited for 1.033004659s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/autoscaling/v2?timeout=32s logger.go:42: 07:26:32 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | test step completed 5-check_redis_config logger.go:42: 07:26:32 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | starting test step 6-check_deployments logger.go:42: 07:26:32 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-repo-server -n $NAMESPACE --template \ '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[uid_entrypoint.sh \ argocd-repo-server \ --redis argocd-redis-ha-haproxy.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/reposerver/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-repo-server deployment is wrong" exit 1 fi ] logger.go:42: 07:26:32 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-server -n $NAMESPACE --template \ '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-server \ --staticassets /shared/app \ --dex-server https://argocd-dex-server.$NAMESPACE.svc.cluster.local:5556 \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --redis argocd-redis-ha-haproxy.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/server/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-server deployment is wrong" exit 1 fi ] logger.go:42: 07:26:35 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | test step completed 6-check_deployments logger.go:42: 07:26:35 | 1-067_validate_redis_secure_comm_no_autotls_ha/7-check_statefulset | starting test step 7-check_statefulset logger.go:42: 07:26:35 | 1-067_validate_redis_secure_comm_no_autotls_ha/7-check_statefulset | running command: [sh -c set -e if test "$(oc get statefulsets.apps argocd-application-controller -n $NAMESPACE --template \ '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-application-controller \ --operation-processors 10 \ --redis argocd-redis-ha-haproxy.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/controller/tls/redis/tls.crt \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --status-processors 20 \ --kubectl-parallelism-limit 10 \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-application-controller statefulsets is wrong" exit 1 fi ] logger.go:42: 07:26:38 | 1-067_validate_redis_secure_comm_no_autotls_ha/7-check_statefulset | test step completed 7-check_statefulset logger.go:42: 07:26:38 | 1-067_validate_redis_secure_comm_no_autotls_ha | skipping kubernetes event logging logger.go:42: 07:26:38 | 1-067_validate_redis_secure_comm_no_autotls_ha | Deleting namespace: kuttl-test-quick-foal === CONT kuttl harness.go:405: run tests finished harness.go:513: cleaning up harness.go:570: removing temp folder: "" --- FAIL: kuttl (2843.12s) --- FAIL: kuttl/harness (0.00s) --- PASS: kuttl/harness/1-083_validate_resource_customization_subkeys (24.83s) --- PASS: kuttl/harness/1-001_alpha_to_beta_dex_conversion (43.42s) --- PASS: kuttl/harness/1-088_validate_applicationset_add_env_vars (52.23s) --- PASS: kuttl/harness/1-089_validate_extra_repo_commands_args (74.43s) --- PASS: kuttl/harness/1-087_validate_repo_server_settings (58.42s) --- PASS: kuttl/harness/1-090_validate_permissions (10.94s) --- PASS: kuttl/harness/1-086_validate_applicationset_extra_command_args (50.95s) --- PASS: kuttl/harness/1-111_validate_appcontroller_initcontainers (13.88s) --- PASS: kuttl/harness/1-084_validate_status_host_ingress (47.20s) --- PASS: kuttl/harness/1-096-validate_home_env_argocd_controller (8.48s) --- PASS: kuttl/harness/1-099_validate_server_autoscale (113.79s) --- PASS: kuttl/harness/1-098_validate_dex_clientsecret_deprecated (35.02s) --- PASS: kuttl/harness/1-114_validate_server_sidecar (13.80s) --- PASS: kuttl/harness/1-091_argocd_app_validate-applicationset-status (21.51s) --- PASS: kuttl/harness/1-113_validate_appcontroller_sidecar (11.55s) --- PASS: kuttl/harness/1-095_validate_dex_clientsecret (52.58s) --- PASS: kuttl/harness/1-112_validate_server_initcontainers (11.27s) --- PASS: kuttl/harness/1-115_validate_controller_extra_command_args (77.70s) --- PASS: kuttl/harness/1-083_validate_kustomize_namereference (43.22s) --- PASS: kuttl/harness/1-048_validate_controller_sharding (87.10s) --- FAIL: kuttl/harness/1-064_validate_security_contexts (13.62s) --- PASS: kuttl/harness/1-107_host_attribute_sso_provider (128.18s) --- PASS: kuttl/harness/1-082_validate_node_placement (19.86s) --- PASS: kuttl/harness/1-081_validate_applicationset_deployment (8.51s) --- PASS: kuttl/harness/1-063_validate_dex_liveness_probe (10.80s) --- PASS: kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount (171.21s) --- FAIL: kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance (13.81s) --- PASS: kuttl/harness/1-063_validate_statefulset_restart (58.22s) --- PASS: kuttl/harness/1-080_validate_regex_support_argocd_rbac (51.76s) --- PASS: kuttl/harness/1-079_validate_vars_for_notificaitons (47.79s) --- PASS: kuttl/harness/1-061_validate_resource_tracking_method (51.59s) --- PASS: kuttl/harness/1-053_validate_cluster_admin_rbac (10.91s) --- PASS: kuttl/harness/1-054_validate_deploymentconfig (94.73s) --- PASS: kuttl/harness/1-058_validate_prometheus_rule (46.95s) --- FAIL: kuttl/harness/1-077_validate_disable_dex_removed (8.75s) --- PASS: kuttl/harness/1-075_validate_dex_anyuid (50.17s) --- PASS: kuttl/harness/1-062_validate_extra_config (183.94s) --- PASS: kuttl/harness/1-052_validate_rolebinding_number (36.67s) --- PASS: kuttl/harness/1-074_validate_terminating_namespace_block (25.07s) --- PASS: kuttl/harness/1-057_validate_notifications (84.32s) --- PASS: kuttl/harness/1-092_validate_workload_status_monitoring_alert (477.85s) --- PASS: kuttl/harness/1-055_validate_notification_controller (49.99s) --- FAIL: kuttl/harness/1-051-validate_csv_permissions (8.86s) --- PASS: kuttl/harness/1-072_validate_liveness_probe_removed (23.32s) --- PASS: kuttl/harness/1-055_drop_resource_customizations (47.75s) --- PASS: kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha (73.73s) --- PASS: kuttl/harness/1-073_validate_rhsso (221.56s) --- PASS: kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha (101.88s) --- PASS: kuttl/harness/1-050_validate_sso (201.40s) --- PASS: kuttl/harness/1-065_validate_redis_ha_anti_affinity (592.26s) --- PASS: kuttl/harness/1-103_argocd_alpha_to_beta_conversion (43.35s) --- PASS: kuttl/harness/1-070_validate_config_management_plugin (89.06s) --- PASS: kuttl/harness/1-102_validate_handle_terminating_namespaces (101.01s) --- PASS: kuttl/harness/1-047_validate_custom_env (56.98s) --- PASS: kuttl/harness/1-038_validate_productized_images (44.86s) --- PASS: kuttl/harness/1-045_validate_repo_exec_timeout (56.49s) --- PASS: kuttl/harness/1-045_validate_applicationset_image_override (18.04s) --- PASS: kuttl/harness/1-033_validate_resource_exclusions (23.76s) --- PASS: kuttl/harness/1-044_validate_resource_limit_changes (58.98s) --- PASS: kuttl/harness/1-043_validate_log_level_format (64.51s) --- PASS: kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount (41.05s) --- PASS: kuttl/harness/1-032_validate_resource_inclusions (28.16s) --- PASS: kuttl/harness/1-043_validate_custom_labels_annotations (16.00s) --- PASS: kuttl/harness/1-049_validate_parallelism_limit (50.13s) --- PASS: kuttl/harness/1-032_validate_dynamic_scaling (302.85s) --- PASS: kuttl/harness/1-037_validate_argocd_setting_replicas (62.04s) --- PASS: kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm (47.93s) --- PASS: kuttl/harness/1-034_validate_webhook_notifications (97.80s) --- PASS: kuttl/harness/1-009_validate-manage-other-namespace (67.43s) --- PASS: kuttl/harness/1-071_validate_SCC_HA (607.25s) --- PASS: kuttl/harness/1-036_validate_keycloak_resource_reqs (191.17s) --- PASS: kuttl/harness/1-030_validate_reencrypt (62.46s) --- PASS: kuttl/harness/1-005_validate_route_tls (128.55s) --- PASS: kuttl/harness/1-007_validate_volume_mounts (50.24s) --- PASS: kuttl/harness/1-007_validate_namespace_scoped_install (45.21s) --- PASS: kuttl/harness/1-021_validate_rolebindings (41.20s) --- PASS: kuttl/harness/1-008_validate-custom-argocd-namespace (75.49s) --- PASS: kuttl/harness/1-019_validate_volume_mounts (43.28s) --- PASS: kuttl/harness/1-029_validate_tls_secret_no_scale (74.79s) --- PASS: kuttl/harness/1-023_validate_repo_server_tls (72.74s) --- PASS: kuttl/harness/1-025-validate-managed-by-change (137.88s) --- PASS: kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion (73.26s) --- PASS: kuttl/harness/1-004_beta_to_alpha_conversion (47.25s) --- PASS: kuttl/harness/1-012_validate-managed-by-chain (133.18s) --- PASS: kuttl/harness/1-003_validate_console_link (8.40s) --- PASS: kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha (1150.37s) --- PASS: kuttl/harness/1-002_alpha_to_beta_keycloak_conversion (127.92s) --- FAIL: kuttl/harness/1-042_restricted_pss_compliant (728.98s) --- PASS: kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha (1168.66s) FAIL Running sequential test suite 2025/02/20 07:27:24 kutt-test config testdirs is overridden with args: [ /kuttl-e2e/gitops-operator/tests/sequential ] === RUN kuttl harness.go:462: starting setup harness.go:252: running tests using configured kubeconfig. I0220 07:27:25.687740 3472 request.go:682] Waited for 1.026549654s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/autoscaling/v2?timeout=32s harness.go:275: Successful connection to cluster at: https://api.nas-psi-414-02.ocp-gitops-qe.com:6443 harness.go:360: running tests harness.go:73: going to run test suite with timeout of 600 seconds for each step harness.go:372: testsuite: /kuttl-e2e/gitops-operator/tests/sequential has 37 tests === RUN kuttl/harness === RUN kuttl/harness/1-002_validate_backend_service === PAUSE kuttl/harness/1-002_validate_backend_service === RUN kuttl/harness/1-003_validate_cluster_config === PAUSE kuttl/harness/1-003_validate_cluster_config === RUN kuttl/harness/1-004_validate_argocd_installation === PAUSE kuttl/harness/1-004_validate_argocd_installation === RUN kuttl/harness/1-005_validate_metrics === PAUSE kuttl/harness/1-005_validate_metrics === RUN kuttl/harness/1-006_validate_machine_config === PAUSE kuttl/harness/1-006_validate_machine_config === RUN kuttl/harness/1-010_validate-ootb-manage-other-namespace === PAUSE kuttl/harness/1-010_validate-ootb-manage-other-namespace === RUN kuttl/harness/1-018_validate_disable_default_instance === PAUSE kuttl/harness/1-018_validate_disable_default_instance === RUN kuttl/harness/1-020_validate_redis_ha_nonha === PAUSE kuttl/harness/1-020_validate_redis_ha_nonha === RUN kuttl/harness/1-026-validate_backend_service_permissions === PAUSE kuttl/harness/1-026-validate_backend_service_permissions === RUN kuttl/harness/1-027_validate_operand_from_git === PAUSE kuttl/harness/1-027_validate_operand_from_git === RUN kuttl/harness/1-028_validate_run_on_infra === PAUSE kuttl/harness/1-028_validate_run_on_infra === RUN kuttl/harness/1-031_validate_toolchain === PAUSE kuttl/harness/1-031_validate_toolchain === RUN kuttl/harness/1-034_validate_custom_roles === PAUSE kuttl/harness/1-034_validate_custom_roles === RUN kuttl/harness/1-035_validate_argocd_secret_repopulate === PAUSE kuttl/harness/1-035_validate_argocd_secret_repopulate === RUN kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace === PAUSE kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace === RUN kuttl/harness/1-037_validate_applicationset_in_any_namespace === PAUSE kuttl/harness/1-037_validate_applicationset_in_any_namespace === RUN kuttl/harness/1-040_validate_quoted_RBAC_group_names === PAUSE kuttl/harness/1-040_validate_quoted_RBAC_group_names === RUN kuttl/harness/1-041_validate_argocd_sync_alert === PAUSE kuttl/harness/1-041_validate_argocd_sync_alert === RUN kuttl/harness/1-042_validate_status_host === PAUSE kuttl/harness/1-042_validate_status_host === RUN kuttl/harness/1-050_validate_sso === PAUSE kuttl/harness/1-050_validate_sso === RUN kuttl/harness/1-056_validate_managed-by === PAUSE kuttl/harness/1-056_validate_managed-by === RUN kuttl/harness/1-064_validate_tcp_reset_error === PAUSE kuttl/harness/1-064_validate_tcp_reset_error === RUN kuttl/harness/1-071_validate_node_selectors === PAUSE kuttl/harness/1-071_validate_node_selectors === RUN kuttl/harness/1-078_validate_default_argocd_consoleLink === PAUSE kuttl/harness/1-078_validate_default_argocd_consoleLink === RUN kuttl/harness/1-083_validate_apps_in_any_namespace === PAUSE kuttl/harness/1-083_validate_apps_in_any_namespace === RUN kuttl/harness/1-084_validate_prune_templates === PAUSE kuttl/harness/1-084_validate_prune_templates === RUN kuttl/harness/1-085_validate_dynamic_plugin_installation === PAUSE kuttl/harness/1-085_validate_dynamic_plugin_installation === RUN kuttl/harness/1-086_validate_default_argocd_role === PAUSE kuttl/harness/1-086_validate_default_argocd_role === RUN kuttl/harness/1-100_validate_rollouts_resources_creation === PAUSE kuttl/harness/1-100_validate_rollouts_resources_creation === RUN kuttl/harness/1-101_validate_rollout_policyrules === PAUSE kuttl/harness/1-101_validate_rollout_policyrules === RUN kuttl/harness/1-104_validate_prometheus_alert === PAUSE kuttl/harness/1-104_validate_prometheus_alert === RUN kuttl/harness/1-105_validate_label_selector === PAUSE kuttl/harness/1-105_validate_label_selector === RUN kuttl/harness/1-106_validate_argocd_metrics_controller === PAUSE kuttl/harness/1-106_validate_argocd_metrics_controller === RUN kuttl/harness/1-109_validate_reencrypt_termination_policy === PAUSE kuttl/harness/1-109_validate_reencrypt_termination_policy === RUN kuttl/harness/1-110_validate_podsecurity_alerts === PAUSE kuttl/harness/1-110_validate_podsecurity_alerts === RUN kuttl/harness/1-111_validate_default_argocd_route === PAUSE kuttl/harness/1-111_validate_default_argocd_route === RUN kuttl/harness/1-112_validate_rollout_plugin_support === PAUSE kuttl/harness/1-112_validate_rollout_plugin_support === CONT kuttl/harness/1-002_validate_backend_service logger.go:42: 07:27:32 | 1-002_validate_backend_service | Creating namespace: kuttl-test-maximum-skink logger.go:42: 07:27:32 | 1-002_validate_backend_service/1-check-backend-service | starting test step 1-check-backend-service logger.go:42: 07:27:34 | 1-002_validate_backend_service/1-check-backend-service | test step completed 1-check-backend-service logger.go:42: 07:27:34 | 1-002_validate_backend_service | skipping kubernetes event logging logger.go:42: 07:27:34 | 1-002_validate_backend_service | Deleting namespace: kuttl-test-maximum-skink === CONT kuttl/harness/1-050_validate_sso logger.go:42: 07:27:40 | 1-050_validate_sso | Creating namespace: kuttl-test-upright-anteater logger.go:42: 07:27:40 | 1-050_validate_sso/1-install | starting test step 1-install I0220 07:27:41.521968 3472 request.go:682] Waited for 1.031545549s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/storage.k8s.io/v1?timeout=32s logger.go:42: 07:27:43 | 1-050_validate_sso/1-install | ArgoCD:kuttl-test-upright-anteater/argocd created logger.go:42: 07:28:15 | 1-050_validate_sso/1-install | test step completed 1-install logger.go:42: 07:28:15 | 1-050_validate_sso/2-remove-spec-sso | starting test step 2-remove-spec-sso logger.go:42: 07:28:15 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso"}]' ] logger.go:42: 07:28:15 | 1-050_validate_sso/2-remove-spec-sso | argocd.argoproj.io/argocd patched logger.go:42: 07:28:15 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c sleep 10] I0220 07:28:26.809150 3472 request.go:682] Waited for 1.007221666s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/migration.k8s.io/v1alpha1?timeout=32s logger.go:42: 07:28:28 | 1-050_validate_sso/2-remove-spec-sso | test step completed 2-remove-spec-sso logger.go:42: 07:28:28 | 1-050_validate_sso/7-provider-keycloak-dex-spec | starting test step 7-provider-keycloak-dex-spec logger.go:42: 07:28:30 | 1-050_validate_sso/7-provider-keycloak-dex-spec | ArgoCD:kuttl-test-upright-anteater/argocd updated logger.go:42: 07:28:31 | 1-050_validate_sso/7-provider-keycloak-dex-spec | test step completed 7-provider-keycloak-dex-spec logger.go:42: 07:28:31 | 1-050_validate_sso/8-remove-spec-dex | starting test step 8-remove-spec-dex logger.go:42: 07:28:31 | 1-050_validate_sso/8-remove-spec-dex | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso/dex"}]' ] logger.go:42: 07:28:31 | 1-050_validate_sso/8-remove-spec-dex | argocd.argoproj.io/argocd patched logger.go:42: 07:30:46 | 1-050_validate_sso/8-remove-spec-dex | test step completed 8-remove-spec-dex logger.go:42: 07:30:46 | 1-050_validate_sso | skipping kubernetes event logging logger.go:42: 07:30:46 | 1-050_validate_sso | Deleting namespace: kuttl-test-upright-anteater === CONT kuttl/harness/1-112_validate_rollout_plugin_support logger.go:42: 07:30:58 | 1-112_validate_rollout_plugin_support | Creating namespace: kuttl-test-heroic-sheep logger.go:42: 07:30:58 | 1-112_validate_rollout_plugin_support/1-install-rolloutmanager | starting test step 1-install-rolloutmanager I0220 07:30:59.311865 3472 request.go:682] Waited for 1.013694686s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1?timeout=32s logger.go:42: 07:31:00 | 1-112_validate_rollout_plugin_support/1-install-rolloutmanager | RolloutManager:openshift-gitops/example-rollout-manager created logger.go:42: 07:31:17 | 1-112_validate_rollout_plugin_support/1-install-rolloutmanager | test step completed 1-install-rolloutmanager logger.go:42: 07:31:17 | 1-112_validate_rollout_plugin_support/3-delete | starting test step 3-delete I0220 07:31:18.484119 3472 request.go:682] Waited for 1.032351566s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/route.openshift.io/v1?timeout=32s logger.go:42: 07:31:19 | 1-112_validate_rollout_plugin_support/3-delete | test step completed 3-delete logger.go:42: 07:31:19 | 1-112_validate_rollout_plugin_support | skipping kubernetes event logging logger.go:42: 07:31:19 | 1-112_validate_rollout_plugin_support | Deleting namespace: kuttl-test-heroic-sheep === CONT kuttl/harness/1-111_validate_default_argocd_route logger.go:42: 07:31:25 | 1-111_validate_default_argocd_route | Creating namespace: kuttl-test-relaxing-teal logger.go:42: 07:31:25 | 1-111_validate_default_argocd_route/1- | starting test step 1- logger.go:42: 07:31:28 | 1-111_validate_default_argocd_route/1- | test step completed 1- logger.go:42: 07:31:28 | 1-111_validate_default_argocd_route/2-update_tls | starting test step 2-update_tls I0220 07:31:29.428350 3472 request.go:682] Waited for 1.033156055s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/rbac.authorization.k8s.io/v1?timeout=32s logger.go:42: 07:31:30 | 1-111_validate_default_argocd_route/2-update_tls | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 07:31:32 | 1-111_validate_default_argocd_route/2-update_tls | test step completed 2-update_tls logger.go:42: 07:31:32 | 1-111_validate_default_argocd_route/3-cleanup | starting test step 3-cleanup logger.go:42: 07:31:32 | 1-111_validate_default_argocd_route/3-cleanup | running command: [sh -c kubectl patch argocd openshift-gitops -n openshift-gitops --type='json' -p='[{"op": "remove", "path": "/spec/server/route/tls"}]' ] logger.go:42: 07:31:49 | 1-111_validate_default_argocd_route/3-cleanup | argocd.argoproj.io/openshift-gitops patched logger.go:42: 07:31:49 | 1-111_validate_default_argocd_route/3-cleanup | running command: [sh -c sleep 10] I0220 07:32:00.988667 3472 request.go:682] Waited for 1.031263071s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha1?timeout=32s logger.go:42: 07:32:02 | 1-111_validate_default_argocd_route/3-cleanup | test step completed 3-cleanup logger.go:42: 07:32:02 | 1-111_validate_default_argocd_route | skipping kubernetes event logging logger.go:42: 07:32:02 | 1-111_validate_default_argocd_route | Deleting namespace: kuttl-test-relaxing-teal === CONT kuttl/harness/1-110_validate_podsecurity_alerts logger.go:42: 07:32:08 | 1-110_validate_podsecurity_alerts | Creating namespace: kuttl-test-climbing-mole logger.go:42: 07:32:08 | 1-110_validate_podsecurity_alerts/1- | starting test step 1- logger.go:42: 07:32:10 | 1-110_validate_podsecurity_alerts/1- | test step completed 1- logger.go:42: 07:32:10 | 1-110_validate_podsecurity_alerts | skipping kubernetes event logging logger.go:42: 07:32:10 | 1-110_validate_podsecurity_alerts | Deleting namespace: kuttl-test-climbing-mole === CONT kuttl/harness/1-109_validate_reencrypt_termination_policy logger.go:42: 07:32:16 | 1-109_validate_reencrypt_termination_policy | Creating namespace: kuttl-test-blessed-catfish logger.go:42: 07:32:16 | 1-109_validate_reencrypt_termination_policy/1- | starting test step 1- I0220 07:32:17.943814 3472 request.go:682] Waited for 1.032750572s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/template.openshift.io/v1?timeout=32s logger.go:42: 07:32:19 | 1-109_validate_reencrypt_termination_policy/1- | test step completed 1- logger.go:42: 07:32:19 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | starting test step 2-check-certificate logger.go:42: 07:32:19 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | running command: [sh -c route=https://$(oc get route -n openshift-gitops openshift-gitops-server -o jsonpath='{.spec.host}') issuer=$(curl --insecure -v "$route" 2>&1 | awk '/issuer:/') if [ $issuer != *"root-ca"* ] && [ $issuer != *"ingress-operator"* ]; then echo "TLS connections rely on the default ingress operator certificate or root-ca" exit 1 fi ] logger.go:42: 07:32:19 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | sh: line 5: [: too many arguments logger.go:42: 07:32:22 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | test step completed 2-check-certificate logger.go:42: 07:32:22 | 1-109_validate_reencrypt_termination_policy | skipping kubernetes event logging logger.go:42: 07:32:22 | 1-109_validate_reencrypt_termination_policy | Deleting namespace: kuttl-test-blessed-catfish === CONT kuttl/harness/1-106_validate_argocd_metrics_controller logger.go:42: 07:32:28 | 1-106_validate_argocd_metrics_controller | Creating namespace: kuttl-test-liberal-kangaroo logger.go:42: 07:32:28 | 1-106_validate_argocd_metrics_controller/1- | starting test step 1- I0220 07:32:29.400463 3472 request.go:682] Waited for 1.032792019s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1?timeout=32s logger.go:42: 07:32:30 | 1-106_validate_argocd_metrics_controller/1- | test step completed 1- logger.go:42: 07:32:30 | 1-106_validate_argocd_metrics_controller/2-disable-metrics | starting test step 2-disable-metrics logger.go:42: 07:32:30 | 1-106_validate_argocd_metrics_controller/2-disable-metrics | running command: [sh -c oc patch argocd openshift-gitops -n openshift-gitops --type='json' -p='[{"op": "replace", "path": "/spec/monitoring/disableMetrics", "value": true}]' ] logger.go:42: 07:33:01 | 1-106_validate_argocd_metrics_controller/2-disable-metrics | Unable to connect to the server: dial tcp 10.0.207.128:6443: i/o timeout I0220 07:33:02.418051 3472 request.go:682] Waited for 1.030966592s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/machine.openshift.io/v1beta1?timeout=32s case.go:364: failed in step 2-disable-metrics case.go:366: exit status 1 logger.go:42: 07:33:03 | 1-106_validate_argocd_metrics_controller | skipping kubernetes event logging logger.go:42: 07:33:03 | 1-106_validate_argocd_metrics_controller | Deleting namespace: kuttl-test-liberal-kangaroo === CONT kuttl/harness/1-105_validate_label_selector logger.go:42: 07:33:09 | 1-105_validate_label_selector | Creating namespace: kuttl-test-picked-ram logger.go:42: 07:33:09 | 1-105_validate_label_selector/1-add-label-selector | starting test step 1-add-label-selector logger.go:42: 07:33:09 | 1-105_validate_label_selector/1-add-label-selector | running command: [sh -c if ! [ -z $NON_OLM ]; then oc set env deployment openshift-gitops-operator-controller-manager -n openshift-gitops-operator ARGOCD_LABEL_SELECTOR=foo=bar -c manager elif ! [ -z $CI ]; then oc patch -n openshift-gitops-operator subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_LABEL_SELECTOR", "value": "foo=bar"}]}}}' else oc patch -n openshift-operators subscription `subscription=gitops-operator && oc get subscription --all-namespaces | grep $subscription | head -1 | awk '{print $2}'` \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_LABEL_SELECTOR", "value": "foo=bar"}]}}}' fi ] logger.go:42: 07:33:11 | 1-105_validate_label_selector/1-add-label-selector | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found logger.go:42: 07:33:11 | 1-105_validate_label_selector/1-add-label-selector | command failure, skipping 1 additional commands I0220 07:33:12.498523 3472 request.go:682] Waited for 1.030355054s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/controlplane.operator.openshift.io/v1alpha1?timeout=32s case.go:364: failed in step 1-add-label-selector case.go:366: exit status 1 logger.go:42: 07:33:13 | 1-105_validate_label_selector | skipping kubernetes event logging logger.go:42: 07:33:13 | 1-105_validate_label_selector | Deleting namespace: kuttl-test-picked-ram === CONT kuttl/harness/1-104_validate_prometheus_alert logger.go:42: 07:33:19 | 1-104_validate_prometheus_alert | Creating namespace: kuttl-test-singular-goose logger.go:42: 07:33:19 | 1-104_validate_prometheus_alert/1- | starting test step 1- logger.go:42: 07:43:22 | 1-104_validate_prometheus_alert/1- | test step failed 1- case.go:364: failed in step 1- case.go:366: servicemonitors.monitoring.coreos.com "openshift-gitops-operator-metrics-monitor" not found logger.go:42: 07:43:22 | 1-104_validate_prometheus_alert | skipping kubernetes event logging logger.go:42: 07:43:22 | 1-104_validate_prometheus_alert | Deleting namespace: kuttl-test-singular-goose === CONT kuttl/harness/1-101_validate_rollout_policyrules logger.go:42: 07:43:28 | 1-101_validate_rollout_policyrules | Creating namespace: kuttl-test-immense-coyote logger.go:42: 07:43:28 | 1-101_validate_rollout_policyrules/1-install-rollloutmanager | starting test step 1-install-rollloutmanager I0220 07:43:29.502990 3472 request.go:682] Waited for 1.032087953s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/infrastructure.cluster.x-k8s.io/v1alpha5?timeout=32s logger.go:42: 07:43:30 | 1-101_validate_rollout_policyrules/1-install-rollloutmanager | RolloutManager:openshift-gitops/example-rollout-manager created logger.go:42: 07:43:32 | 1-101_validate_rollout_policyrules/1-install-rollloutmanager | test step completed 1-install-rollloutmanager logger.go:42: 07:43:32 | 1-101_validate_rollout_policyrules/2-delete | starting test step 2-delete logger.go:42: 07:43:34 | 1-101_validate_rollout_policyrules/2-delete | test step completed 2-delete logger.go:42: 07:43:34 | 1-101_validate_rollout_policyrules | skipping kubernetes event logging logger.go:42: 07:43:34 | 1-101_validate_rollout_policyrules | Deleting namespace: kuttl-test-immense-coyote === CONT kuttl/harness/1-100_validate_rollouts_resources_creation logger.go:42: 07:43:40 | 1-100_validate_rollouts_resources_creation | Creating namespace: kuttl-test-large-prawn logger.go:42: 07:43:40 | 1-100_validate_rollouts_resources_creation/1-install-rolloutmanager | starting test step 1-install-rolloutmanager I0220 07:43:41.620239 3472 request.go:682] Waited for 1.032000493s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/samples.operator.openshift.io/v1?timeout=32s logger.go:42: 07:43:43 | 1-100_validate_rollouts_resources_creation/1-install-rolloutmanager | RolloutManager:openshift-gitops/example-rollout-manager created logger.go:42: 07:43:58 | 1-100_validate_rollouts_resources_creation/1-install-rolloutmanager | test step completed 1-install-rolloutmanager logger.go:42: 07:43:58 | 1-100_validate_rollouts_resources_creation/2- | starting test step 2- I0220 07:43:59.729114 3472 request.go:682] Waited for 1.031960373s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/discovery.k8s.io/v1?timeout=32s logger.go:42: 07:44:01 | 1-100_validate_rollouts_resources_creation/2- | test step completed 2- logger.go:42: 07:44:01 | 1-100_validate_rollouts_resources_creation/3-delete | starting test step 3-delete logger.go:42: 07:44:03 | 1-100_validate_rollouts_resources_creation/3-delete | test step completed 3-delete logger.go:42: 07:44:03 | 1-100_validate_rollouts_resources_creation | skipping kubernetes event logging logger.go:42: 07:44:03 | 1-100_validate_rollouts_resources_creation | Deleting namespace: kuttl-test-large-prawn === CONT kuttl/harness/1-086_validate_default_argocd_role logger.go:42: 07:44:09 | 1-086_validate_default_argocd_role | Creating namespace: kuttl-test-able-skunk logger.go:42: 07:44:09 | 1-086_validate_default_argocd_role/1- | starting test step 1- I0220 07:44:10.868899 3472 request.go:682] Waited for 1.03013353s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s logger.go:42: 07:44:12 | 1-086_validate_default_argocd_role/1- | test step completed 1- logger.go:42: 07:44:12 | 1-086_validate_default_argocd_role/2-install | starting test step 2-install logger.go:42: 07:44:14 | 1-086_validate_default_argocd_role/2-install | Namespace:/test-1-086-custom created logger.go:42: 07:44:14 | 1-086_validate_default_argocd_role/2-install | Namespace:/test-1-086-custom2 created logger.go:42: 07:44:14 | 1-086_validate_default_argocd_role/2-install | Namespace:/test-1-086-custom3 created logger.go:42: 07:44:15 | 1-086_validate_default_argocd_role/2-install | ArgoCD:test-1-086-custom/argocd-default-policy created logger.go:42: 07:44:15 | 1-086_validate_default_argocd_role/2-install | ArgoCD:test-1-086-custom2/argocd-default-policy-empty created logger.go:42: 07:44:15 | 1-086_validate_default_argocd_role/2-install | ArgoCD:test-1-086-custom3/argocd-default-policy-admin created logger.go:42: 07:44:48 | 1-086_validate_default_argocd_role/2-install | test step completed 2-install logger.go:42: 07:44:48 | 1-086_validate_default_argocd_role | skipping kubernetes event logging logger.go:42: 07:44:48 | 1-086_validate_default_argocd_role | Deleting namespace: kuttl-test-able-skunk === CONT kuttl/harness/1-085_validate_dynamic_plugin_installation logger.go:42: 07:44:54 | 1-085_validate_dynamic_plugin_installation | Creating namespace: kuttl-test-unbiased-tadpole logger.go:42: 07:44:54 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | starting test step 1-install-dynamic-plugin logger.go:42: 07:44:54 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | running command: [sh -c set -eo pipefail # Define constants export subscription="openshift-gitops-operator" ns=$(oc get subscription openshift-gitops-operator -n openshift-operators -o jsonpath='{.metadata.namespace}' --ignore-not-found) installation_ns=${ns:-openshift-gitops-operator} # Get OCP version echo "Detecting OpenShift Container Platform (OCP) version..." ocp_version=$(oc version | awk '/Server Version/ {print $3}' | cut -d '.' -f1,2) echo "Detected OCP version: ${ocp_version}" if [[ ${ocp_version} > '4.15' ]]; then echo "OCP version ${ocp_version} does not require manual enabling of the GitOps plugin." exit 0 fi # Check if the gitops-plugin Deployment exists and is ready plugin_status=$(oc get deployment/gitops-plugin -n openshift-gitops -o jsonpath='{.status.readyReplicas}' 2>/dev/null || echo "0") if [ "$plugin_status" -ge 1 ]; then echo "GitOps plugin is already enabled and running." exit 0 fi echo "GitOps plugin is not enabled. Proceeding to enable it." # Patch the Subscription to add the necessary env key:value for Dynamic Plugin echo "Appending the required environment variable to 'env'." oc patch subscription "${subscription}" -n $installation_ns \ --type='json' \ -p='[{ "op": "add", "path": "/spec/config", "value": { "env": [{ "name": "DYNAMIC_PLUGIN_START_OCP_VERSION", "value": "'${ocp_version}'" }] } }]' echo "Successfully patched the subscription to enable the GitOps plugin." # Allow resources to stabilize before the next step sleep 5 ] logger.go:42: 07:44:55 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | Detecting OpenShift Container Platform (OCP) version... logger.go:42: 07:44:55 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | Detected OCP version: 4.13 logger.go:42: 07:44:55 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | GitOps plugin is not enabled. Proceeding to enable it. logger.go:42: 07:44:55 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | Appending the required environment variable to 'env'. logger.go:42: 07:44:56 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 07:44:56 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | Successfully patched the subscription to enable the GitOps plugin. I0220 07:45:02.436077 3472 request.go:682] Waited for 1.031848476s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/metal3.io/v1alpha1?timeout=32s logger.go:42: 07:45:03 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | test step completed 1-install-dynamic-plugin logger.go:42: 07:45:03 | 1-085_validate_dynamic_plugin_installation/2- | starting test step 2- logger.go:42: 07:45:30 | 1-085_validate_dynamic_plugin_installation/2- | test step completed 2- logger.go:42: 07:45:30 | 1-085_validate_dynamic_plugin_installation | skipping kubernetes event logging logger.go:42: 07:45:30 | 1-085_validate_dynamic_plugin_installation | Deleting namespace: kuttl-test-unbiased-tadpole === CONT kuttl/harness/1-084_validate_prune_templates 2025/02/20 07:45:36 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/01-assert.yaml 2025/02/20 07:45:36 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/01-create-ns-managed-by.yaml 2025/02/20 07:45:36 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/02-prune-duplicateGVK-resource.yaml 2025/02/20 07:45:36 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/03-errors.yaml 2025/02/20 07:45:36 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/04-delete.yaml logger.go:42: 07:45:36 | 1-084_validate_prune_templates | Creating namespace: kuttl-test-humble-husky logger.go:42: 07:45:36 | 1-084_validate_prune_templates/1-create-ns-managed-by | starting test step 1-create-ns-managed-by I0220 07:45:37.431636 3472 request.go:682] Waited for 1.032577058s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/config.openshift.io/v1?timeout=32s logger.go:42: 07:45:38 | 1-084_validate_prune_templates/1-create-ns-managed-by | test step completed 1-create-ns-managed-by logger.go:42: 07:45:38 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | starting test step 2-prune-duplicateGVK-resource logger.go:42: 07:45:41 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | test step completed 2-prune-duplicateGVK-resource logger.go:42: 07:45:41 | 1-084_validate_prune_templates/3- | starting test step 3- logger.go:42: 07:45:43 | 1-084_validate_prune_templates/3- | test step completed 3- logger.go:42: 07:45:43 | 1-084_validate_prune_templates/4-delete | starting test step 4-delete logger.go:42: 07:45:46 | 1-084_validate_prune_templates/4-delete | test step completed 4-delete logger.go:42: 07:45:46 | 1-084_validate_prune_templates | skipping kubernetes event logging logger.go:42: 07:45:46 | 1-084_validate_prune_templates | Deleting namespace: kuttl-test-humble-husky === CONT kuttl/harness/1-083_validate_apps_in_any_namespace logger.go:42: 07:45:52 | 1-083_validate_apps_in_any_namespace | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 07:45:52 | 1-083_validate_apps_in_any_namespace | Creating namespace: kuttl-test-full-hippo logger.go:42: 07:45:52 | 1-083_validate_apps_in_any_namespace/1-create-ns-managed-by | starting test step 1-create-ns-managed-by I0220 07:45:53.348515 3472 request.go:682] Waited for 1.032777646s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/security.openshift.io/v1?timeout=32s logger.go:42: 07:45:54 | 1-083_validate_apps_in_any_namespace/1-create-ns-managed-by | Namespace:/test-1-24-custom created logger.go:42: 07:45:55 | 1-083_validate_apps_in_any_namespace/1-create-ns-managed-by | test step completed 1-create-ns-managed-by logger.go:42: 07:45:55 | 1-083_validate_apps_in_any_namespace/2-add-source-ns | starting test step 2-add-source-ns logger.go:42: 07:45:57 | 1-083_validate_apps_in_any_namespace/2-add-source-ns | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 07:45:57 | 1-083_validate_apps_in_any_namespace/2-add-source-ns | test step completed 2-add-source-ns logger.go:42: 07:45:57 | 1-083_validate_apps_in_any_namespace/3-delete | starting test step 3-delete I0220 07:46:04.615324 3472 request.go:682] Waited for 1.032645651s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s logger.go:42: 07:46:06 | 1-083_validate_apps_in_any_namespace/3-delete | test step completed 3-delete logger.go:42: 07:46:06 | 1-083_validate_apps_in_any_namespace/4-change-source-namespaces | starting test step 4-change-source-namespaces logger.go:42: 07:46:08 | 1-083_validate_apps_in_any_namespace/4-change-source-namespaces | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 07:46:09 | 1-083_validate_apps_in_any_namespace/4-change-source-namespaces | test step completed 4-change-source-namespaces logger.go:42: 07:46:09 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | starting test step 5-create-ns-and-add-source-ns logger.go:42: 07:46:12 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | Namespace:/test-1-24-custom created logger.go:42: 07:46:12 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 07:46:43 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | test step completed 5-create-ns-and-add-source-ns logger.go:42: 07:46:43 | 1-083_validate_apps_in_any_namespace/6-add-managed-by | starting test step 6-add-managed-by I0220 07:46:44.571431 3472 request.go:682] Waited for 1.03258932s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/config.openshift.io/v1?timeout=32s logger.go:42: 07:46:46 | 1-083_validate_apps_in_any_namespace/6-add-managed-by | Namespace:/test-1-24-custom updated logger.go:42: 07:46:47 | 1-083_validate_apps_in_any_namespace/6-add-managed-by | test step completed 6-add-managed-by logger.go:42: 07:46:47 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | starting test step 7-remove-managed-by logger.go:42: 07:46:47 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c oc label ns test-1-24-custom argocd.argoproj.io/managed-by-] logger.go:42: 07:46:47 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | namespace/test-1-24-custom unlabeled logger.go:42: 07:46:47 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c oc rollout restart deployment.apps/openshift-gitops-server -n openshift-gitops] logger.go:42: 07:46:48 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | deployment.apps/openshift-gitops-server restarted logger.go:42: 07:46:48 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c oc rollout restart statefulset.apps/openshift-gitops-application-controller -n openshift-gitops] logger.go:42: 07:46:48 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | statefulset.apps/openshift-gitops-application-controller restarted logger.go:42: 07:46:48 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c sleep 5] I0220 07:46:54.677513 3472 request.go:682] Waited for 1.031590204s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1?timeout=32s logger.go:42: 07:47:18 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | test step completed 7-remove-managed-by logger.go:42: 07:47:18 | 1-083_validate_apps_in_any_namespace/8-change-source-namespaces | starting test step 8-change-source-namespaces I0220 07:47:19.770743 3472 request.go:682] Waited for 1.032896476s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/discovery.k8s.io/v1?timeout=32s logger.go:42: 07:47:21 | 1-083_validate_apps_in_any_namespace/8-change-source-namespaces | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 07:47:22 | 1-083_validate_apps_in_any_namespace/8-change-source-namespaces | test step completed 8-change-source-namespaces logger.go:42: 07:47:22 | 1-083_validate_apps_in_any_namespace/99-delete | starting test step 99-delete I0220 07:47:29.816833 3472 request.go:682] Waited for 1.532764124s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha1?timeout=32s logger.go:42: 07:47:30 | 1-083_validate_apps_in_any_namespace/99-delete | test step completed 99-delete logger.go:42: 07:47:30 | 1-083_validate_apps_in_any_namespace | skipping kubernetes event logging logger.go:42: 07:47:30 | 1-083_validate_apps_in_any_namespace | Deleting namespace: kuttl-test-full-hippo === CONT kuttl/harness/1-078_validate_default_argocd_consoleLink logger.go:42: 07:47:36 | 1-078_validate_default_argocd_consoleLink | Creating namespace: kuttl-test-concise-quail logger.go:42: 07:47:36 | 1-078_validate_default_argocd_consoleLink/1- | starting test step 1- logger.go:42: 07:47:39 | 1-078_validate_default_argocd_consoleLink/1- | test step completed 1- logger.go:42: 07:47:39 | 1-078_validate_default_argocd_consoleLink/2-add_env | starting test step 2-add_env logger.go:42: 07:47:39 | 1-078_validate_default_argocd_consoleLink/2-add_env | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "DISABLE_DEFAULT_ARGOCD_CONSOLELINK", "value": "true"}]}}}' ] logger.go:42: 07:47:39 | 1-078_validate_default_argocd_consoleLink/2-add_env | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found I0220 07:47:40.582265 3472 request.go:682] Waited for 1.032678843s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apps.openshift.io/v1?timeout=32s case.go:364: failed in step 2-add_env case.go:366: exit status 1 logger.go:42: 07:47:42 | 1-078_validate_default_argocd_consoleLink | skipping kubernetes event logging logger.go:42: 07:47:42 | 1-078_validate_default_argocd_consoleLink | Deleting namespace: kuttl-test-concise-quail === CONT kuttl/harness/1-071_validate_node_selectors logger.go:42: 07:47:47 | 1-071_validate_node_selectors | Creating namespace: kuttl-test-evident-mole logger.go:42: 07:47:47 | 1-071_validate_node_selectors/1- | starting test step 1- logger.go:42: 07:47:50 | 1-071_validate_node_selectors/1- | test step completed 1- logger.go:42: 07:47:50 | 1-071_validate_node_selectors/2-custom-node-selector | starting test step 2-custom-node-selector logger.go:42: 07:47:50 | 1-071_validate_node_selectors/2-custom-node-selector | running command: [sh -c oc patch gitopsservice cluster --type=merge --patch='{"spec": {"nodeSelector": {"key1": "value1"}}}' ] logger.go:42: 07:47:50 | 1-071_validate_node_selectors/2-custom-node-selector | gitopsservice.pipelines.openshift.io/cluster patched I0220 07:47:51.897031 3472 request.go:682] Waited for 1.032355419s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/user.openshift.io/v1?timeout=32s logger.go:42: 07:47:53 | 1-071_validate_node_selectors/2-custom-node-selector | test step completed 2-custom-node-selector logger.go:42: 07:47:53 | 1-071_validate_node_selectors/3-patch-run-on-infra | starting test step 3-patch-run-on-infra logger.go:42: 07:47:53 | 1-071_validate_node_selectors/3-patch-run-on-infra | running command: [sh -c oc patch gitopsservice cluster --type=merge --patch='{"spec": {"runOnInfra": true, "tolerations": [{"effect": "NoSchedule", "key": "infra", "value": "reserved"}]}}' ] logger.go:42: 07:47:53 | 1-071_validate_node_selectors/3-patch-run-on-infra | gitopsservice.pipelines.openshift.io/cluster patched logger.go:42: 07:47:56 | 1-071_validate_node_selectors/3-patch-run-on-infra | test step completed 3-patch-run-on-infra logger.go:42: 07:47:56 | 1-071_validate_node_selectors/4-change-back | starting test step 4-change-back logger.go:42: 07:47:56 | 1-071_validate_node_selectors/4-change-back | running command: [sh -c oc patch gitopsservice cluster --type=json --patch='[{"op": "remove", "path": "/spec/tolerations"}, {"op": "remove", "path": "/spec/runOnInfra"}, {"op": "remove", "path": "/spec/nodeSelector"}]' oc patch statefulset openshift-gitops-application-controller -n openshift-gitops -p '{"spec":{"replicas":0}}' ] logger.go:42: 07:47:56 | 1-071_validate_node_selectors/4-change-back | gitopsservice.pipelines.openshift.io/cluster patched logger.go:42: 07:47:57 | 1-071_validate_node_selectors/4-change-back | statefulset.apps/openshift-gitops-application-controller patched logger.go:42: 07:47:59 | 1-071_validate_node_selectors/4-change-back | test step completed 4-change-back logger.go:42: 07:47:59 | 1-071_validate_node_selectors | skipping kubernetes event logging logger.go:42: 07:47:59 | 1-071_validate_node_selectors | Deleting namespace: kuttl-test-evident-mole === CONT kuttl/harness/1-064_validate_tcp_reset_error logger.go:42: 07:48:05 | 1-064_validate_tcp_reset_error | Creating namespace: kuttl-test-expert-porpoise logger.go:42: 07:48:05 | 1-064_validate_tcp_reset_error/1- | starting test step 1- I0220 07:48:06.649856 3472 request.go:682] Waited for 1.033275956s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s logger.go:42: 07:48:08 | 1-064_validate_tcp_reset_error/1- | test step completed 1- logger.go:42: 07:48:08 | 1-064_validate_tcp_reset_error/2-install | starting test step 2-install logger.go:42: 07:48:10 | 1-064_validate_tcp_reset_error/2-install | Application:openshift-gitops/1-27-argocd created logger.go:42: 07:48:48 | 1-064_validate_tcp_reset_error/2-install | test step completed 2-install logger.go:42: 07:48:48 | 1-064_validate_tcp_reset_error/3-create-app | starting test step 3-create-app I0220 07:48:49.710309 3472 request.go:682] Waited for 1.033291564s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/image.openshift.io/v1?timeout=32s logger.go:42: 07:48:51 | 1-064_validate_tcp_reset_error/3-create-app | Application:test-1-27-custom/guestbook created logger.go:42: 07:49:00 | 1-064_validate_tcp_reset_error/3-create-app | test step completed 3-create-app logger.go:42: 07:49:00 | 1-064_validate_tcp_reset_error/4-check-app-manifest | starting test step 4-check-app-manifest logger.go:42: 07:49:00 | 1-064_validate_tcp_reset_error/4-check-app-manifest | running command: [sh -c set -e cleanup() { argocd logout "${api_server}" unset api_server unset password } trap cleanup INT TERM EXIT # Get credentials api_server=$(oc get routes -n openshift-gitops --field-selector metadata.name=openshift-gitops-server -o jsonpath="{.items[*]['spec.host']}") password=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d) # Login into ArgoCD login=$(argocd login $api_server --username admin --password $password --insecure) if ! [[ "${login}" =~ "'admin:login' logged in successfully" ]]; then echo "ERROR: unable to login to ArgoCD remote api server" exit 1 fi # Execute 'argocd app manifest' and check the 'read tcp' error app_manifests=$(argocd app manifests 1-27-argocd --source git --revision HEAD) if [[ "${app_manifests}" =~ 'Original error: read tcp' ]]; then echo "ERROR: TCP reset error is present in this code" exit 2 fi exit 0 ] logger.go:42: 07:49:01 | 1-064_validate_tcp_reset_error/4-check-app-manifest | time="2025-02-20T07:49:01Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 07:49:03 | 1-064_validate_tcp_reset_error/4-check-app-manifest | time="2025-02-20T07:49:03Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 07:49:03 | 1-064_validate_tcp_reset_error/4-check-app-manifest | Logged out from 'openshift-gitops-server-openshift-gitops.apps.nas-psi-414-02.ocp-gitops-qe.com' I0220 07:49:04.345476 3472 request.go:682] Waited for 1.03306411s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/migration.k8s.io/v1alpha1?timeout=32s logger.go:42: 07:49:05 | 1-064_validate_tcp_reset_error/4-check-app-manifest | test step completed 4-check-app-manifest logger.go:42: 07:49:05 | 1-064_validate_tcp_reset_error/99-delete | starting test step 99-delete I0220 07:49:18.844102 3472 request.go:682] Waited for 1.03198759s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/oauth.openshift.io/v1?timeout=32s logger.go:42: 07:49:20 | 1-064_validate_tcp_reset_error/99-delete | test step completed 99-delete logger.go:42: 07:49:20 | 1-064_validate_tcp_reset_error | skipping kubernetes event logging logger.go:42: 07:49:20 | 1-064_validate_tcp_reset_error | Deleting namespace: kuttl-test-expert-porpoise === CONT kuttl/harness/1-056_validate_managed-by logger.go:42: 07:49:26 | 1-056_validate_managed-by | Creating namespace: kuttl-test-accepted-marten logger.go:42: 07:49:26 | 1-056_validate_managed-by/1-install | starting test step 1-install logger.go:42: 07:49:28 | 1-056_validate_managed-by/1-install | Namespace:/test-1-56-target created logger.go:42: 07:49:28 | 1-056_validate_managed-by/1-install | Namespace:/test-1-56-custom created logger.go:42: 07:49:28 | 1-056_validate_managed-by/1-install | ArgoCD:test-1-56-target/example-argocd created logger.go:42: 07:50:01 | 1-056_validate_managed-by/1-install | test step completed 1-install logger.go:42: 07:50:01 | 1-056_validate_managed-by/2-delete | starting test step 2-delete I0220 07:50:02.272130 3472 request.go:682] Waited for 1.031810423s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 07:50:03 | 1-056_validate_managed-by/2-delete | test step completed 2-delete logger.go:42: 07:50:03 | 1-056_validate_managed-by/3-check | starting test step 3-check logger.go:42: 07:50:03 | 1-056_validate_managed-by/3-check | running command: [sh -c set -e label_value=$(oc get ns/test-1-56-custom -o go-template=$'{{index .metadata.labels "argocd.argoproj.io/managed-by"}}') echo "label value: ${label_value}" if [[ "${label_value}" == "test-1-56-target" ]]; then echo "managed-by label exists, set REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION variable to delete the label" exit 0 fi exit 1 ] logger.go:42: 07:50:04 | 1-056_validate_managed-by/3-check | label value: test-1-56-target logger.go:42: 07:50:04 | 1-056_validate_managed-by/3-check | managed-by label exists, set REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION variable to delete the label logger.go:42: 07:50:06 | 1-056_validate_managed-by/3-check | test step completed 3-check logger.go:42: 07:50:06 | 1-056_validate_managed-by/4-add_env | starting test step 4-add_env logger.go:42: 07:50:06 | 1-056_validate_managed-by/4-add_env | running command: [sh -c set -e oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION", "value": "true"}]}}}' ] logger.go:42: 07:50:06 | 1-056_validate_managed-by/4-add_env | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found case.go:364: failed in step 4-add_env case.go:366: exit status 1 logger.go:42: 07:50:09 | 1-056_validate_managed-by | skipping kubernetes event logging logger.go:42: 07:50:09 | 1-056_validate_managed-by | Deleting namespace: kuttl-test-accepted-marten === CONT kuttl/harness/1-028_validate_run_on_infra logger.go:42: 07:50:15 | 1-028_validate_run_on_infra | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 07:50:15 | 1-028_validate_run_on_infra | Creating namespace: kuttl-test-peaceful-griffon logger.go:42: 07:50:15 | 1-028_validate_run_on_infra/1-install | starting test step 1-install logger.go:42: 07:50:15 | 1-028_validate_run_on_infra/1-install | running command: [sh -c oc patch gitopsservice cluster --type=merge --patch='{"spec": {"runOnInfra": true, "tolerations": [{"effect": "NoSchedule", "key": "infra", "value": "reserved"}]}}' ] logger.go:42: 07:50:15 | 1-028_validate_run_on_infra/1-install | gitopsservice.pipelines.openshift.io/cluster patched I0220 07:50:16.997499 3472 request.go:682] Waited for 1.032397738s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apps.openshift.io/v1?timeout=32s logger.go:42: 07:50:18 | 1-028_validate_run_on_infra/1-install | test step completed 1-install logger.go:42: 07:50:18 | 1-028_validate_run_on_infra/2-custom-argocd | starting test step 2-custom-argocd logger.go:42: 07:50:21 | 1-028_validate_run_on_infra/2-custom-argocd | ArgoCD:kuttl-test-peaceful-griffon/argocd created logger.go:42: 07:50:53 | 1-028_validate_run_on_infra/2-custom-argocd | test step completed 2-custom-argocd logger.go:42: 07:50:53 | 1-028_validate_run_on_infra/3-change-back | starting test step 3-change-back logger.go:42: 07:50:53 | 1-028_validate_run_on_infra/3-change-back | running command: [sh -c oc patch gitopsservice cluster --type=json --patch='[{"op": "remove", "path": "/spec/tolerations"}, {"op": "remove", "path": "/spec/runOnInfra"}]' oc patch statefulset openshift-gitops-application-controller -n openshift-gitops -p '{"spec":{"replicas":0}}' ] logger.go:42: 07:50:53 | 1-028_validate_run_on_infra/3-change-back | gitopsservice.pipelines.openshift.io/cluster patched logger.go:42: 07:50:54 | 1-028_validate_run_on_infra/3-change-back | statefulset.apps/openshift-gitops-application-controller patched I0220 07:50:55.096665 3472 request.go:682] Waited for 1.025629299s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/security.openshift.io/v1?timeout=32s logger.go:42: 07:50:56 | 1-028_validate_run_on_infra/3-change-back | test step completed 3-change-back logger.go:42: 07:50:56 | 1-028_validate_run_on_infra/4-check | starting test step 4-check logger.go:42: 07:50:59 | 1-028_validate_run_on_infra/4-check | test step completed 4-check logger.go:42: 07:50:59 | 1-028_validate_run_on_infra | skipping kubernetes event logging logger.go:42: 07:50:59 | 1-028_validate_run_on_infra | Deleting namespace: kuttl-test-peaceful-griffon === CONT kuttl/harness/1-042_validate_status_host logger.go:42: 07:51:05 | 1-042_validate_status_host | Creating namespace: kuttl-test-known-cougar logger.go:42: 07:51:05 | 1-042_validate_status_host/1-install | starting test step 1-install I0220 07:51:06.364146 3472 request.go:682] Waited for 1.031193492s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/cloud.network.openshift.io/v1?timeout=32s logger.go:42: 07:51:07 | 1-042_validate_status_host/1-install | ArgoCD:kuttl-test-known-cougar/example-argocd created logger.go:42: 07:51:39 | 1-042_validate_status_host/1-install | test step completed 1-install logger.go:42: 07:51:39 | 1-042_validate_status_host/2-check_URL | starting test step 2-check_URL logger.go:42: 07:51:39 | 1-042_validate_status_host/2-check_URL | running command: [sh -c until [[ $i -eq 20 ]] do route_URL=$(oc get routes -o jsonpath="{.items[*]['spec.host']}" --field-selector metadata.name=example-argocd-server -n $NAMESPACE) status_URL=$(oc get argocd -o jsonpath="{.items[*]['status.host']}" -n $NAMESPACE) echo -e "route_URL:${route_URL}\nstatus_URL:${status_URL}" if ! [[ "${status_URL}" == "${route_URL}" ]]; then i=$((i+1)) sleep 6 else exit 0 fi done echo "Error: Route does not match" exit 1 ] logger.go:42: 07:51:40 | 1-042_validate_status_host/2-check_URL | route_URL:example-argocd-server-kuttl-test-known-cougar.apps.nas-psi-414-02.ocp-gitops-qe.com logger.go:42: 07:51:40 | 1-042_validate_status_host/2-check_URL | status_URL:example-argocd-server-kuttl-test-known-cougar.apps.nas-psi-414-02.ocp-gitops-qe.com I0220 07:51:41.760636 3472 request.go:682] Waited for 1.032864813s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s logger.go:42: 07:51:43 | 1-042_validate_status_host/2-check_URL | test step completed 2-check_URL logger.go:42: 07:51:43 | 1-042_validate_status_host/3-modify_route_URL | starting test step 3-modify_route_URL logger.go:42: 07:51:43 | 1-042_validate_status_host/3-modify_route_URL | running command: [sh -c oc patch route example-argocd-server -n $NAMESPACE --type='json' -p='[{"op": "replace", "path": "/spec/host", "value":"modified-route"}]' ] logger.go:42: 07:51:43 | 1-042_validate_status_host/3-modify_route_URL | route.route.openshift.io/example-argocd-server patched logger.go:42: 07:51:46 | 1-042_validate_status_host/3-modify_route_URL | test step completed 3-modify_route_URL logger.go:42: 07:51:46 | 1-042_validate_status_host/4-disable_route | starting test step 4-disable_route logger.go:42: 07:51:48 | 1-042_validate_status_host/4-disable_route | ArgoCD:kuttl-test-known-cougar/example-argocd updated logger.go:42: 07:51:48 | 1-042_validate_status_host/4-disable_route | test step completed 4-disable_route logger.go:42: 07:51:48 | 1-042_validate_status_host/5-check_URL_gone | starting test step 5-check_URL_gone logger.go:42: 07:51:48 | 1-042_validate_status_host/5-check_URL_gone | running command: [sh -c sleep 5 status_URL=$(oc get argocd -o jsonpath="{.items[*]['status.host']}" -n $NAMESPACE) echo "${status_URL}" if test "${status_URL}" != ""; then echo "Error: status.host should not be present in argocd" exit 1 fi exit 0 ] logger.go:42: 07:51:54 | 1-042_validate_status_host/5-check_URL_gone | I0220 07:51:55.215805 3472 request.go:682] Waited for 1.03228313s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/metal3.io/v1alpha1?timeout=32s logger.go:42: 07:51:56 | 1-042_validate_status_host/5-check_URL_gone | test step completed 5-check_URL_gone logger.go:42: 07:51:56 | 1-042_validate_status_host | skipping kubernetes event logging logger.go:42: 07:51:56 | 1-042_validate_status_host | Deleting namespace: kuttl-test-known-cougar === CONT kuttl/harness/1-041_validate_argocd_sync_alert logger.go:42: 07:52:02 | 1-041_validate_argocd_sync_alert | Creating namespace: kuttl-test-brave-malamute logger.go:42: 07:52:02 | 1-041_validate_argocd_sync_alert/1- | starting test step 1- I0220 07:52:05.253799 3472 request.go:682] Waited for 2.481046824s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/config.openshift.io/v1?timeout=32s logger.go:42: 07:52:05 | 1-041_validate_argocd_sync_alert/1- | test step completed 1- logger.go:42: 07:52:05 | 1-041_validate_argocd_sync_alert | skipping kubernetes event logging logger.go:42: 07:52:05 | 1-041_validate_argocd_sync_alert | Deleting namespace: kuttl-test-brave-malamute === CONT kuttl/harness/1-040_validate_quoted_RBAC_group_names logger.go:42: 07:52:11 | 1-040_validate_quoted_RBAC_group_names | Creating namespace: kuttl-test-quiet-crane logger.go:42: 07:52:11 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | starting test step 1-login_argocd_api_server logger.go:42: 07:52:11 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | running command: [sh -c api_server=$(oc get routes -n openshift-gitops --field-selector metadata.name=openshift-gitops-server -o jsonpath="{.items[*]['spec.host']}") password=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d) output=$(argocd login $api_server --username admin --password $password --insecure) if ! [[ "${output}" =~ "'admin:login' logged in successfully" ]]; then exit 1 fi exit 0 ] logger.go:42: 07:52:12 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | time="2025-02-20T07:52:12Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." I0220 07:52:15.293823 3472 request.go:682] Waited for 2.281986618s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v2?timeout=32s logger.go:42: 07:52:15 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | test step completed 1-login_argocd_api_server logger.go:42: 07:52:15 | 1-040_validate_quoted_RBAC_group_names/2-create_role | starting test step 2-create_role logger.go:42: 07:52:15 | 1-040_validate_quoted_RBAC_group_names/2-create_role | running command: [sh -c output=$(argocd proj role create default somerole) if ! [[ "$output" =~ "Role 'somerole' created" ]]; then echo "Error creating role" exit 1 fi exit 0 ] logger.go:42: 07:52:15 | 1-040_validate_quoted_RBAC_group_names/2-create_role | time="2025-02-20T07:52:15Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 07:52:18 | 1-040_validate_quoted_RBAC_group_names/2-create_role | test step completed 2-create_role logger.go:42: 07:52:18 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | starting test step 3-specify_group_names logger.go:42: 07:52:18 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | running command: [sh -c output=$(argocd proj role add-group default somerole '"CN=foo,OU=bar,O=baz"') cleanup() { argocd proj role delete default somerole } trap cleanup INT TERM EXIT if ! [[ "${output}" =~ "added to role 'somerole'" ]]; then echo "Contains comma characters!" exit 1 fi ] logger.go:42: 07:52:18 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | time="2025-02-20T07:52:18Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 07:52:18 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | time="2025-02-20T07:52:18Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 07:52:19 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | Role 'somerole' deleted logger.go:42: 07:52:21 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | test step completed 3-specify_group_names logger.go:42: 07:52:21 | 1-040_validate_quoted_RBAC_group_names | skipping kubernetes event logging logger.go:42: 07:52:21 | 1-040_validate_quoted_RBAC_group_names | Deleting namespace: kuttl-test-quiet-crane === CONT kuttl/harness/1-037_validate_applicationset_in_any_namespace logger.go:42: 07:52:27 | 1-037_validate_applicationset_in_any_namespace | Creating namespace: kuttl-test-classic-mako logger.go:42: 07:52:27 | 1-037_validate_applicationset_in_any_namespace/0-prepare | starting test step 0-prepare I0220 07:52:28.482519 3472 request.go:682] Waited for 1.033258282s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s logger.go:42: 07:52:29 | 1-037_validate_applicationset_in_any_namespace/0-prepare | Namespace:/appset-argocd created logger.go:42: 07:52:30 | 1-037_validate_applicationset_in_any_namespace/0-prepare | Namespace:/appset-old-ns created logger.go:42: 07:52:30 | 1-037_validate_applicationset_in_any_namespace/0-prepare | Namespace:/appset-new-ns created logger.go:42: 07:52:30 | 1-037_validate_applicationset_in_any_namespace/0-prepare | test step completed 0-prepare logger.go:42: 07:52:30 | 1-037_validate_applicationset_in_any_namespace/1-argocd | starting test step 1-argocd logger.go:42: 07:52:32 | 1-037_validate_applicationset_in_any_namespace/1-argocd | ArgoCD:appset-argocd/example created logger.go:42: 07:53:05 | 1-037_validate_applicationset_in_any_namespace/1-argocd | test step completed 1-argocd logger.go:42: 07:53:05 | 1-037_validate_applicationset_in_any_namespace/2-enable-appset-in-any-ns | starting test step 2-enable-appset-in-any-ns I0220 07:53:06.411769 3472 request.go:682] Waited for 1.030754364s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/events.k8s.io/v1?timeout=32s logger.go:42: 07:53:07 | 1-037_validate_applicationset_in_any_namespace/2-enable-appset-in-any-ns | ArgoCD:appset-argocd/example updated logger.go:42: 07:53:07 | 1-037_validate_applicationset_in_any_namespace/2-enable-appset-in-any-ns | test step completed 2-enable-appset-in-any-ns logger.go:42: 07:53:07 | 1-037_validate_applicationset_in_any_namespace/3-add-new-appset-ns | starting test step 3-add-new-appset-ns logger.go:42: 07:53:10 | 1-037_validate_applicationset_in_any_namespace/3-add-new-appset-ns | ArgoCD:appset-argocd/example updated logger.go:42: 07:53:10 | 1-037_validate_applicationset_in_any_namespace/3-add-new-appset-ns | test step completed 3-add-new-appset-ns logger.go:42: 07:53:10 | 1-037_validate_applicationset_in_any_namespace/4-enable-apps-in-new-ns | starting test step 4-enable-apps-in-new-ns logger.go:42: 07:53:13 | 1-037_validate_applicationset_in_any_namespace/4-enable-apps-in-new-ns | ArgoCD:appset-argocd/example updated logger.go:42: 07:53:14 | 1-037_validate_applicationset_in_any_namespace/4-enable-apps-in-new-ns | test step completed 4-enable-apps-in-new-ns logger.go:42: 07:53:14 | 1-037_validate_applicationset_in_any_namespace/5-enable-apps-in-old-ns | starting test step 5-enable-apps-in-old-ns I0220 07:53:16.411813 3472 request.go:682] Waited for 1.928760564s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/migration.k8s.io/v1alpha1?timeout=32s logger.go:42: 07:53:17 | 1-037_validate_applicationset_in_any_namespace/5-enable-apps-in-old-ns | ArgoCD:appset-argocd/example updated logger.go:42: 07:53:18 | 1-037_validate_applicationset_in_any_namespace/5-enable-apps-in-old-ns | test step completed 5-enable-apps-in-old-ns logger.go:42: 07:53:18 | 1-037_validate_applicationset_in_any_namespace/6-disable-appset-in-new-ns | starting test step 6-disable-appset-in-new-ns logger.go:42: 07:53:20 | 1-037_validate_applicationset_in_any_namespace/6-disable-appset-in-new-ns | ArgoCD:appset-argocd/example updated logger.go:42: 07:53:22 | 1-037_validate_applicationset_in_any_namespace/6-disable-appset-in-new-ns | test step completed 6-disable-appset-in-new-ns logger.go:42: 07:53:22 | 1-037_validate_applicationset_in_any_namespace/7-disable-apps-in-all-ns | starting test step 7-disable-apps-in-all-ns logger.go:42: 07:53:24 | 1-037_validate_applicationset_in_any_namespace/7-disable-apps-in-all-ns | ArgoCD:appset-argocd/example updated logger.go:42: 07:53:25 | 1-037_validate_applicationset_in_any_namespace/7-disable-apps-in-all-ns | test step completed 7-disable-apps-in-all-ns logger.go:42: 07:53:25 | 1-037_validate_applicationset_in_any_namespace/99-cleanup | starting test step 99-cleanup I0220 07:54:01.627782 3472 request.go:682] Waited for 1.03159298s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha2?timeout=32s logger.go:42: 07:54:03 | 1-037_validate_applicationset_in_any_namespace/99-cleanup | test step completed 99-cleanup logger.go:42: 07:54:03 | 1-037_validate_applicationset_in_any_namespace | skipping kubernetes event logging logger.go:42: 07:54:03 | 1-037_validate_applicationset_in_any_namespace | Deleting namespace: kuttl-test-classic-mako === CONT kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace logger.go:42: 07:54:09 | 1-036_validate_role_rolebinding_for_source_namespace | Creating namespace: kuttl-test-noble-haddock logger.go:42: 07:54:09 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | starting test step 1-sourcenamespace_without_wildcard I0220 07:54:11.655999 3472 request.go:682] Waited for 2.430497649s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1alpha1?timeout=32s logger.go:42: 07:54:11 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | Namespace:/test created logger.go:42: 07:54:11 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | ArgoCD:default/example-argocd created logger.go:42: 07:54:12 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | test step completed 1-sourcenamespace_without_wildcard logger.go:42: 07:54:12 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | starting test step 2-sourceNamespace_with_wildcard_pattern logger.go:42: 07:54:15 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | Namespace:/test-1 created logger.go:42: 07:54:15 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | Namespace:/dev created logger.go:42: 07:54:15 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | ArgoCD:default/example-argocd updated logger.go:42: 07:54:16 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | test step completed 2-sourceNamespace_with_wildcard_pattern logger.go:42: 07:54:16 | 1-036_validate_role_rolebinding_for_source_namespace/3-new_namespace_with_match_pattern | starting test step 3-new_namespace_with_match_pattern logger.go:42: 07:54:19 | 1-036_validate_role_rolebinding_for_source_namespace/3-new_namespace_with_match_pattern | Namespace:/test-2 created logger.go:42: 07:54:20 | 1-036_validate_role_rolebinding_for_source_namespace/3-new_namespace_with_match_pattern | test step completed 3-new_namespace_with_match_pattern logger.go:42: 07:54:20 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | starting test step 4-sourceNamespace_with_wildcard I0220 07:54:21.698720 3472 request.go:682] Waited for 1.382518995s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 07:54:22 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | Namespace:/test-3 created logger.go:42: 07:54:22 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | ArgoCD:default/example-argocd updated logger.go:42: 07:54:26 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | test step completed 4-sourceNamespace_with_wildcard logger.go:42: 07:54:26 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | starting test step 5-multiple_sourcenamespace logger.go:42: 07:54:29 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | Namespace:/test-ns-1 created logger.go:42: 07:54:29 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | Namespace:/dev-ns-1 created logger.go:42: 07:54:29 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | Namespace:/other-ns created logger.go:42: 07:54:29 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | ArgoCD:default/example-argocd updated logger.go:42: 07:54:30 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | test step completed 5-multiple_sourcenamespace logger.go:42: 07:54:30 | 1-036_validate_role_rolebinding_for_source_namespace/6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace | starting test step 6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace I0220 07:54:31.712382 3472 request.go:682] Waited for 1.283197189s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1alpha1?timeout=32s logger.go:42: 07:54:32 | 1-036_validate_role_rolebinding_for_source_namespace/6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace | ArgoCD:default/example-argocd updated logger.go:42: 07:54:35 | 1-036_validate_role_rolebinding_for_source_namespace/6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace | test step completed 6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace logger.go:42: 07:54:35 | 1-036_validate_role_rolebinding_for_source_namespace | skipping kubernetes event logging logger.go:42: 07:54:35 | 1-036_validate_role_rolebinding_for_source_namespace | Deleting namespace: kuttl-test-noble-haddock === CONT kuttl/harness/1-035_validate_argocd_secret_repopulate logger.go:42: 07:54:41 | 1-035_validate_argocd_secret_repopulate | Creating namespace: kuttl-test-epic-grub logger.go:42: 07:54:41 | 1-035_validate_argocd_secret_repopulate/1- | starting test step 1- I0220 07:54:42.443985 3472 request.go:682] Waited for 1.032798755s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/apiserver.openshift.io/v1?timeout=32s logger.go:42: 07:54:43 | 1-035_validate_argocd_secret_repopulate/1- | test step completed 1- logger.go:42: 07:54:43 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | starting test step 2-remove_secret_data logger.go:42: 07:54:43 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | running command: [sh -c oc patch secret/argocd-secret -p '{"data":null}' -n openshift-gitops ] logger.go:42: 07:54:44 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | secret/argocd-secret patched logger.go:42: 07:54:46 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | test step completed 2-remove_secret_data logger.go:42: 07:54:46 | 1-035_validate_argocd_secret_repopulate/3-check_secret_repopulate | starting test step 3-check_secret_repopulate logger.go:42: 07:54:46 | 1-035_validate_argocd_secret_repopulate/3-check_secret_repopulate | running command: [sh -c until [[ $i -eq 20 ]] do secret_data=$(oc get secret/argocd-secret -n openshift-gitops -o custom-columns=SECRET:.data --no-headers) if [[ $secret_data == "" ]] then i=$((i+1)) sleep 6 else echo "argocd-secret was repopulated" exit 0 fi done echo "argocd-secret was not repopulated" exit 1 ] logger.go:42: 07:54:47 | 1-035_validate_argocd_secret_repopulate/3-check_secret_repopulate | argocd-secret was repopulated logger.go:42: 07:54:49 | 1-035_validate_argocd_secret_repopulate/3-check_secret_repopulate | test step completed 3-check_secret_repopulate logger.go:42: 07:54:49 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | starting test step 4-check_controller_pod_status logger.go:42: 07:54:49 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | running command: [sh -c oc rollout status deployment openshift-gitops-operator-controller-manager -n openshift-operators --timeout=1m rollout_status=$? if [ "${rollout_status}" -ne 0 ]; then echo "openshift-gitops-operator-controller-manager deployment was not successfully rolled out" exit 1 fi exit 0 ] logger.go:42: 07:54:50 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | Error from server (NotFound): deployments.apps "openshift-gitops-operator-controller-manager" not found logger.go:42: 07:54:50 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | openshift-gitops-operator-controller-manager deployment was not successfully rolled out I0220 07:54:52.483492 3472 request.go:682] Waited for 2.433402502s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/packages.operators.coreos.com/v1?timeout=32s case.go:364: failed in step 4-check_controller_pod_status case.go:366: exit status 1 logger.go:42: 07:54:52 | 1-035_validate_argocd_secret_repopulate | skipping kubernetes event logging logger.go:42: 07:54:52 | 1-035_validate_argocd_secret_repopulate | Deleting namespace: kuttl-test-epic-grub === CONT kuttl/harness/1-034_validate_custom_roles logger.go:42: 07:54:58 | 1-034_validate_custom_roles | Creating namespace: kuttl-test-informed-lynx logger.go:42: 07:54:58 | 1-034_validate_custom_roles/1-install | starting test step 1-install logger.go:42: 07:55:00 | 1-034_validate_custom_roles/1-install | Namespace:/test-1-034-custom created logger.go:42: 07:55:00 | 1-034_validate_custom_roles/1-install | ArgoCD:test-1-034-custom/argocd created logger.go:42: 07:55:01 | 1-034_validate_custom_roles/1-install | Namespace:/custom-role-namespace created logger.go:42: 07:55:01 | 1-034_validate_custom_roles/1-install | ClusterRole:/custom-argocd-role created logger.go:42: 07:55:13 | 1-034_validate_custom_roles/1-install | test step completed 1-install logger.go:42: 07:55:13 | 1-034_validate_custom_roles/2-patch-subscription | starting test step 2-patch-subscription logger.go:42: 07:55:13 | 1-034_validate_custom_roles/2-patch-subscription | running command: [sh -c if ! [ -z $NON_OLM ]; then oc set env deployment openshift-gitops-operator-controller-manager -n openshift-gitops-operator CONTROLLER_CLUSTER_ROLE=custom-argocd-role oc set env deployment openshift-gitops-operator-controller-manager -n openshift-gitops-operator SERVER_CLUSTER_ROLE=custom-argocd-role elif ! [ -z $CI ]; then oc patch -n openshift-gitops-operator subscription `subscription=gitops-operator- && oc get subscription --all-namespaces | grep $subscription | head -1 | awk '{print $2}'`\ --type merge --patch '{"spec": {"config": {"env": [{"name": "CONTROLLER_CLUSTER_ROLE", "value": "custom-argocd-role"},{"name": "SERVER_CLUSTER_ROLE", "value": "custom-argocd-role"}]}}}' else oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "CONTROLLER_CLUSTER_ROLE", "value": "custom-argocd-role"},{"name": "SERVER_CLUSTER_ROLE", "value": "custom-argocd-role"}]}}}' fi ] logger.go:42: 07:55:13 | 1-034_validate_custom_roles/2-patch-subscription | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found logger.go:42: 07:55:13 | 1-034_validate_custom_roles/2-patch-subscription | command failure, skipping 1 additional commands I0220 07:55:14.978391 3472 request.go:682] Waited for 1.032880416s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s case.go:364: failed in step 2-patch-subscription case.go:366: exit status 1 logger.go:42: 07:55:16 | 1-034_validate_custom_roles | skipping kubernetes event logging logger.go:42: 07:55:16 | 1-034_validate_custom_roles | Deleting namespace: kuttl-test-informed-lynx === CONT kuttl/harness/1-031_validate_toolchain logger.go:42: 07:55:22 | 1-031_validate_toolchain | Creating namespace: kuttl-test-settling-cub logger.go:42: 07:55:22 | 1-031_validate_toolchain/1-check | starting test step 1-check logger.go:42: 07:55:22 | 1-031_validate_toolchain/1-check | running command: [sh -c set -e set -o pipefail # These variables need to be maintained according to the component matrix: https://spaces.redhat.com/display/GITOPS/GitOps+Component+Matrix expected_kustomizeVersion='v5.4.3' expected_helmVersion='v3.15.4' expected_argocdVersion='v2.13.5' expected_dexVersion='v2.41.1-1-ga7854d65' expected_redisVersion='6.2.17' gitops_server_pod=$(oc get -n openshift-gitops pods \ -o custom-columns=NAME:.metadata.name --no-headers | \ grep openshift-gitops-server | \ head -1 ) dex_pod=$(oc get -n openshift-gitops pods \ -o custom-columns=NAME:.metadata.name --no-headers | \ grep openshift-gitops-dex-server | \ head -1 ) redis_pod=$(oc get -n openshift-gitops pods \ -o custom-columns=NAME:.metadata.name --no-headers | \ grep openshift-gitops-redis | \ head -1 ) route=$(oc get route -n openshift-gitops \ -o custom-columns=NAME:.spec.host --no-headers | \ grep openshift-gitops-server | \ head -1 ) kustomizeVersion=$(oc -n openshift-gitops exec $gitops_server_pod \ -- kustomize version ) helmVersion=$(oc -n openshift-gitops exec $gitops_server_pod \ -- helm version | \ sed -e 's/version.BuildInfo//' -e 's/\"//g' | \ awk -F':' '{ print $2 }' | \ awk -F',' '{ print $1 }' ) argocdVersion=$(oc -n openshift-gitops exec $gitops_server_pod \ -- argocd version --short --server $route --insecure | \ grep 'argocd-server' | \ awk -F' ' '{ print $2 }' | \ sed 's/\+.*//' ) dexVersion=$(oc -n openshift-gitops exec $dex_pod -- dex version 2>&1 | \ grep '^Dex Version' | \ awk -F': ' '{ print $2 }' ) redisVersion=$(oc -n openshift-gitops exec $redis_pod -- \ redis-server -v | awk -F '=' '{ print $2 }' | cut -d' ' -f 1 ) if test "${kustomizeVersion}" != "${expected_kustomizeVersion}"; then echo "Kustomize version mismatch. Should be ${expected_kustomizeVersion}, is ${kustomizeVersion}" exit 1 fi if test "${helmVersion}" != "${expected_helmVersion}"; then echo "Helm version mismatch. Should be ${expected_helmVersion}, is ${helmVersion}" exit 1 fi if test "${argocdVersion}" != "${expected_argocdVersion}"; then echo "ArgoCD version mismatch. Should be ${expected_argocdVersion}, is ${argocdVersion}" exit 1 fi if test "${dexVersion}" != "${expected_dexVersion}"; then echo "Dex version mismatch. Should be ${expected_dexVersion}, is ${dexVersion}" exit 1 fi if test "${redisVersion}" != "${expected_redisVersion}"; then echo "Redis version mismatch. Should be ${expected_redisVersion}, is ${redisVersion}" exit 1 fi ] logger.go:42: 07:55:26 | 1-031_validate_toolchain/1-check | time="2025-02-20T07:55:26Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." I0220 07:55:28.363307 3472 request.go:682] Waited for 1.032159836s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 07:55:29 | 1-031_validate_toolchain/1-check | test step completed 1-check logger.go:42: 07:55:29 | 1-031_validate_toolchain | skipping kubernetes event logging logger.go:42: 07:55:29 | 1-031_validate_toolchain | Deleting namespace: kuttl-test-settling-cub === CONT kuttl/harness/1-026-validate_backend_service_permissions logger.go:42: 07:55:35 | 1-026-validate_backend_service_permissions | Creating namespace: kuttl-test-relevant-gopher logger.go:42: 07:55:35 | 1-026-validate_backend_service_permissions/1-check-backend-service | starting test step 1-check-backend-service logger.go:42: 07:55:38 | 1-026-validate_backend_service_permissions/1-check-backend-service | test step completed 1-check-backend-service logger.go:42: 07:55:38 | 1-026-validate_backend_service_permissions | skipping kubernetes event logging logger.go:42: 07:55:38 | 1-026-validate_backend_service_permissions | Deleting namespace: kuttl-test-relevant-gopher === CONT kuttl/harness/1-027_validate_operand_from_git logger.go:42: 07:55:44 | 1-027_validate_operand_from_git | Creating namespace: kuttl-test-concrete-crawdad logger.go:42: 07:55:44 | 1-027_validate_operand_from_git/1- | starting test step 1- I0220 07:55:45.176624 3472 request.go:682] Waited for 1.033011995s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/argoproj.io/v1beta1?timeout=32s logger.go:42: 07:55:46 | 1-027_validate_operand_from_git/1- | test step completed 1- logger.go:42: 07:55:46 | 1-027_validate_operand_from_git/2-install | starting test step 2-install logger.go:42: 07:55:49 | 1-027_validate_operand_from_git/2-install | Application:openshift-gitops/1-27-argocd created logger.go:42: 07:56:26 | 1-027_validate_operand_from_git/2-install | test step completed 2-install logger.go:42: 07:56:26 | 1-027_validate_operand_from_git/3-create-app | starting test step 3-create-app I0220 07:56:27.978390 3472 request.go:682] Waited for 1.03199588s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/coordination.k8s.io/v1?timeout=32s logger.go:42: 07:56:29 | 1-027_validate_operand_from_git/3-create-app | Application:test-1-27-custom/guestbook created logger.go:42: 07:56:36 | 1-027_validate_operand_from_git/3-create-app | test step completed 3-create-app logger.go:42: 07:56:36 | 1-027_validate_operand_from_git/99-delete | starting test step 99-delete I0220 07:56:49.136064 3472 request.go:682] Waited for 1.028345688s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s logger.go:42: 07:56:50 | 1-027_validate_operand_from_git/99-delete | test step completed 99-delete logger.go:42: 07:56:50 | 1-027_validate_operand_from_git | skipping kubernetes event logging logger.go:42: 07:56:50 | 1-027_validate_operand_from_git | Deleting namespace: kuttl-test-concrete-crawdad === CONT kuttl/harness/1-020_validate_redis_ha_nonha logger.go:42: 07:56:56 | 1-020_validate_redis_ha_nonha | Creating namespace: kuttl-test-fine-bream logger.go:42: 07:56:56 | 1-020_validate_redis_ha_nonha/1- | starting test step 1- logger.go:42: 07:56:59 | 1-020_validate_redis_ha_nonha/1- | test step completed 1- logger.go:42: 07:56:59 | 1-020_validate_redis_ha_nonha/2-enable-ha | starting test step 2-enable-ha I0220 07:57:00.115529 3472 request.go:682] Waited for 1.032276632s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/whereabouts.cni.cncf.io/v1alpha1?timeout=32s logger.go:42: 07:57:01 | 1-020_validate_redis_ha_nonha/2-enable-ha | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 08:06:05 | 1-020_validate_redis_ha_nonha/2-enable-ha | test step completed 2-enable-ha logger.go:42: 08:06:05 | 1-020_validate_redis_ha_nonha/3-update-ha-resources | starting test step 3-update-ha-resources I0220 08:06:06.344644 3472 request.go:682] Waited for 1.03239459s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 08:06:07 | 1-020_validate_redis_ha_nonha/3-update-ha-resources | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 08:06:55 | 1-020_validate_redis_ha_nonha/3-update-ha-resources | test step completed 3-update-ha-resources logger.go:42: 08:06:55 | 1-020_validate_redis_ha_nonha/4-disable-ha | starting test step 4-disable-ha I0220 08:06:56.078669 3472 request.go:682] Waited for 1.032020773s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 08:06:57 | 1-020_validate_redis_ha_nonha/4-disable-ha | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 08:07:10 | 1-020_validate_redis_ha_nonha/4-disable-ha | test step completed 4-disable-ha logger.go:42: 08:07:10 | 1-020_validate_redis_ha_nonha | skipping kubernetes event logging logger.go:42: 08:07:10 | 1-020_validate_redis_ha_nonha | Deleting namespace: kuttl-test-fine-bream === CONT kuttl/harness/1-005_validate_metrics logger.go:42: 08:07:16 | 1-005_validate_metrics | Creating namespace: kuttl-test-capable-reindeer logger.go:42: 08:07:16 | 1-005_validate_metrics/1- | starting test step 1- I0220 08:07:17.276131 3472 request.go:682] Waited for 1.030784322s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1?timeout=32s logger.go:42: 08:07:18 | 1-005_validate_metrics/1- | test step completed 1- logger.go:42: 08:07:18 | 1-005_validate_metrics | skipping kubernetes event logging logger.go:42: 08:07:18 | 1-005_validate_metrics | Deleting namespace: kuttl-test-capable-reindeer === CONT kuttl/harness/1-006_validate_machine_config logger.go:42: 08:07:24 | 1-006_validate_machine_config | Creating namespace: kuttl-test-classic-stinkbug logger.go:42: 08:07:24 | 1-006_validate_machine_config/1-install | starting test step 1-install I0220 08:07:27.284610 3472 request.go:682] Waited for 2.432594842s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s logger.go:42: 08:07:27 | 1-006_validate_machine_config/1-install | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 08:07:27 | 1-006_validate_machine_config/1-install | Application:openshift-gitops/validate-machine-config created logger.go:42: 08:07:38 | 1-006_validate_machine_config/1-install | test step completed 1-install logger.go:42: 08:07:38 | 1-006_validate_machine_config/2- | starting test step 2- I0220 08:07:39.609655 3472 request.go:682] Waited for 1.03229292s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/argoproj.io/v1alpha1?timeout=32s logger.go:42: 08:07:41 | 1-006_validate_machine_config/2- | test step completed 2- logger.go:42: 08:07:41 | 1-006_validate_machine_config/3-scale-down-repo-server | starting test step 3-scale-down-repo-server logger.go:42: 08:07:43 | 1-006_validate_machine_config/3-scale-down-repo-server | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 08:07:44 | 1-006_validate_machine_config/3-scale-down-repo-server | test step completed 3-scale-down-repo-server logger.go:42: 08:07:44 | 1-006_validate_machine_config/99-delete | starting test step 99-delete logger.go:42: 08:07:47 | 1-006_validate_machine_config/99-delete | test step completed 99-delete logger.go:42: 08:07:47 | 1-006_validate_machine_config | skipping kubernetes event logging logger.go:42: 08:07:47 | 1-006_validate_machine_config | Deleting namespace: kuttl-test-classic-stinkbug === CONT kuttl/harness/1-004_validate_argocd_installation logger.go:42: 08:07:53 | 1-004_validate_argocd_installation | Creating namespace: kuttl-test-cunning-hare logger.go:42: 08:07:53 | 1-004_validate_argocd_installation/1-check-prereqs | starting test step 1-check-prereqs I0220 08:07:54.375554 3472 request.go:682] Waited for 1.031536784s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/controlplane.operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 08:07:55 | 1-004_validate_argocd_installation/1-check-prereqs | test step completed 1-check-prereqs logger.go:42: 08:07:55 | 1-004_validate_argocd_installation/2-change-operand | starting test step 2-change-operand logger.go:42: 08:07:58 | 1-004_validate_argocd_installation/2-change-operand | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 08:07:58 | 1-004_validate_argocd_installation/2-change-operand | test step completed 2-change-operand logger.go:42: 08:07:58 | 1-004_validate_argocd_installation/3-wait | starting test step 3-wait logger.go:42: 08:07:58 | 1-004_validate_argocd_installation/3-wait | running command: [sh -c sleep 5] I0220 08:08:04.488096 3472 request.go:682] Waited for 1.032013463s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/config.openshift.io/v1?timeout=32s logger.go:42: 08:08:05 | 1-004_validate_argocd_installation/3-wait | test step completed 3-wait logger.go:42: 08:08:05 | 1-004_validate_argocd_installation/4- | starting test step 4- logger.go:42: 08:08:10 | 1-004_validate_argocd_installation/4- | test step completed 4- logger.go:42: 08:08:10 | 1-004_validate_argocd_installation | skipping kubernetes event logging logger.go:42: 08:08:10 | 1-004_validate_argocd_installation | Deleting namespace: kuttl-test-cunning-hare === CONT kuttl/harness/1-003_validate_cluster_config logger.go:42: 08:08:16 | 1-003_validate_cluster_config | Creating namespace: kuttl-test-intent-cardinal logger.go:42: 08:08:16 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | starting test step 0-cluster-scoped-argocd I0220 08:08:17.426811 3472 request.go:682] Waited for 1.032019681s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s logger.go:42: 08:08:18 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | Namespace:/argocd-e2e-cluster-config created logger.go:42: 08:08:18 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | ArgoCD:argocd-e2e-cluster-config/example-argocd created logger.go:42: 08:08:18 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | test step completed 0-cluster-scoped-argocd logger.go:42: 08:08:18 | 1-003_validate_cluster_config/1-cluster-scoped | starting test step 1-cluster-scoped logger.go:42: 08:08:18 | 1-003_validate_cluster_config/1-cluster-scoped | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_CLUSTER_CONFIG_NAMESPACES", "value": "openshift-gitops, argocd-e2e-cluster-config"}]}}}' ] logger.go:42: 08:08:19 | 1-003_validate_cluster_config/1-cluster-scoped | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found case.go:364: failed in step 1-cluster-scoped case.go:366: exit status 1 logger.go:42: 08:08:22 | 1-003_validate_cluster_config | skipping kubernetes event logging logger.go:42: 08:08:22 | 1-003_validate_cluster_config | Deleting namespace: kuttl-test-intent-cardinal === CONT kuttl/harness/1-018_validate_disable_default_instance logger.go:42: 08:08:27 | 1-018_validate_disable_default_instance | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 08:08:27 | 1-018_validate_disable_default_instance | Creating namespace: kuttl-test-amused-porpoise logger.go:42: 08:08:27 | 1-018_validate_disable_default_instance/1- | starting test step 1- I0220 08:08:28.867236 3472 request.go:682] Waited for 1.033139471s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/metal3.io/v1alpha1?timeout=32s logger.go:42: 08:08:30 | 1-018_validate_disable_default_instance/1- | test step completed 1- logger.go:42: 08:08:30 | 1-018_validate_disable_default_instance/2-patch-subscription | starting test step 2-patch-subscription logger.go:42: 08:08:30 | 1-018_validate_disable_default_instance/2-patch-subscription | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "DISABLE_DEFAULT_ARGOCD_INSTANCE", "value": "true"}]}}}' ] logger.go:42: 08:08:30 | 1-018_validate_disable_default_instance/2-patch-subscription | Error from server (NotFound): subscriptions.operators.coreos.com "openshift-gitops-operator" not found logger.go:42: 08:08:30 | 1-018_validate_disable_default_instance/2-patch-subscription | command failure, skipping 1 additional commands case.go:364: failed in step 2-patch-subscription case.go:366: exit status 1 logger.go:42: 08:08:33 | 1-018_validate_disable_default_instance | skipping kubernetes event logging logger.go:42: 08:08:33 | 1-018_validate_disable_default_instance | Deleting namespace: kuttl-test-amused-porpoise === CONT kuttl/harness/1-010_validate-ootb-manage-other-namespace logger.go:42: 08:08:38 | 1-010_validate-ootb-manage-other-namespace | Ignoring README.md as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 08:08:38 | 1-010_validate-ootb-manage-other-namespace | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 08:08:38 | 1-010_validate-ootb-manage-other-namespace | Creating namespace: kuttl-test-sharp-quetzal logger.go:42: 08:08:39 | 1-010_validate-ootb-manage-other-namespace/1-install | starting test step 1-install I0220 08:08:40.051670 3472 request.go:682] Waited for 1.032559945s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/build.openshift.io/v1?timeout=32s logger.go:42: 08:08:41 | 1-010_validate-ootb-manage-other-namespace/1-install | Namespace:/test-1-10-custom created logger.go:42: 08:08:41 | 1-010_validate-ootb-manage-other-namespace/1-install | test step completed 1-install logger.go:42: 08:08:41 | 1-010_validate-ootb-manage-other-namespace/2- | starting test step 2- logger.go:42: 08:08:44 | 1-010_validate-ootb-manage-other-namespace/2- | test step completed 2- logger.go:42: 08:08:44 | 1-010_validate-ootb-manage-other-namespace/3-create-application | starting test step 3-create-application logger.go:42: 08:08:46 | 1-010_validate-ootb-manage-other-namespace/3-create-application | Application:openshift-gitops/test-1-10-custom created logger.go:42: 08:08:54 | 1-010_validate-ootb-manage-other-namespace/3-create-application | test step completed 3-create-application logger.go:42: 08:08:54 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | starting test step 4-unlabel-namespace logger.go:42: 08:08:54 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | running command: [sh -c kubectl label ns test-1-10-custom argocd.argoproj.io/managed-by-] logger.go:42: 08:08:55 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | namespace/test-1-10-custom unlabeled logger.go:42: 08:08:55 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | running command: [sh -c sleep 5] I0220 08:09:01.205483 3472 request.go:682] Waited for 1.032735702s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/snapshot.storage.k8s.io/v1?timeout=32s logger.go:42: 08:09:02 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | test step completed 4-unlabel-namespace logger.go:42: 08:09:02 | 1-010_validate-ootb-manage-other-namespace/5-check-rbac | starting test step 5-check-rbac logger.go:42: 08:09:05 | 1-010_validate-ootb-manage-other-namespace/5-check-rbac | test step completed 5-check-rbac logger.go:42: 08:09:05 | 1-010_validate-ootb-manage-other-namespace/99-delete | starting test step 99-delete I0220 08:09:17.852725 3472 request.go:682] Waited for 1.03145982s due to client-side throttling, not priority and fairness, request: GET:https://api.nas-psi-414-02.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1?timeout=32s logger.go:42: 08:09:19 | 1-010_validate-ootb-manage-other-namespace/99-delete | test step completed 99-delete logger.go:42: 08:09:19 | 1-010_validate-ootb-manage-other-namespace | skipping kubernetes event logging logger.go:42: 08:09:19 | 1-010_validate-ootb-manage-other-namespace | Deleting namespace: kuttl-test-sharp-quetzal === CONT kuttl harness.go:405: run tests finished harness.go:513: cleaning up harness.go:570: removing temp folder: "" --- FAIL: kuttl (2520.58s) --- FAIL: kuttl/harness (0.00s) --- PASS: kuttl/harness/1-002_validate_backend_service (8.31s) --- PASS: kuttl/harness/1-050_validate_sso (197.79s) --- PASS: kuttl/harness/1-112_validate_rollout_plugin_support (27.56s) --- PASS: kuttl/harness/1-111_validate_default_argocd_route (42.59s) --- PASS: kuttl/harness/1-110_validate_podsecurity_alerts (8.48s) --- PASS: kuttl/harness/1-109_validate_reencrypt_termination_policy (11.46s) --- FAIL: kuttl/harness/1-106_validate_argocd_metrics_controller (41.38s) --- FAIL: kuttl/harness/1-105_validate_label_selector (10.08s) --- FAIL: kuttl/harness/1-104_validate_prometheus_alert (608.62s) --- PASS: kuttl/harness/1-101_validate_rollout_policyrules (12.13s) --- PASS: kuttl/harness/1-100_validate_rollouts_resources_creation (29.25s) --- PASS: kuttl/harness/1-086_validate_default_argocd_role (45.10s) --- PASS: kuttl/harness/1-085_validate_dynamic_plugin_installation (41.45s) --- PASS: kuttl/harness/1-084_validate_prune_templates (15.91s) --- PASS: kuttl/harness/1-083_validate_apps_in_any_namespace (104.39s) --- FAIL: kuttl/harness/1-078_validate_default_argocd_consoleLink (11.22s) --- PASS: kuttl/harness/1-071_validate_node_selectors (17.70s) --- PASS: kuttl/harness/1-064_validate_tcp_reset_error (80.58s) --- FAIL: kuttl/harness/1-056_validate_managed-by (49.32s) --- PASS: kuttl/harness/1-028_validate_run_on_infra (49.82s) --- PASS: kuttl/harness/1-042_validate_status_host (57.44s) --- PASS: kuttl/harness/1-041_validate_argocd_sync_alert (8.39s) --- PASS: kuttl/harness/1-040_validate_quoted_RBAC_group_names (16.27s) --- PASS: kuttl/harness/1-037_validate_applicationset_in_any_namespace (101.77s) --- PASS: kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace (32.16s) --- FAIL: kuttl/harness/1-035_validate_argocd_secret_repopulate (17.04s) --- FAIL: kuttl/harness/1-034_validate_custom_roles (23.86s) --- PASS: kuttl/harness/1-031_validate_toolchain (13.44s) --- PASS: kuttl/harness/1-026-validate_backend_service_permissions (8.42s) --- PASS: kuttl/harness/1-027_validate_operand_from_git (72.35s) --- PASS: kuttl/harness/1-020_validate_redis_ha_nonha (619.76s) --- PASS: kuttl/harness/1-005_validate_metrics (8.61s) --- PASS: kuttl/harness/1-006_validate_machine_config (28.49s) --- PASS: kuttl/harness/1-004_validate_argocd_installation (23.05s) --- FAIL: kuttl/harness/1-003_validate_cluster_config (11.44s) --- FAIL: kuttl/harness/1-018_validate_disable_default_instance (11.18s) --- PASS: kuttl/harness/1-010_validate-ootb-manage-other-namespace (46.19s) FAIL ========================== ==== E2E TESTS FAILED ==== ========================== Copying results Deleted temp working directory /kuttl-e2e/gitops-operator/tmp.FMt4PZvtnK make: *** [Makefile:4: e2e-tests] Error 1 make: Leaving directory '/kuttl-e2e/gitops-operator' File has been uploaded to CI/z-stream-test-pipeline-nas/250220-063841/sequential_2025.02.20-06.39.59.xml 🚀 File has been uploaded to CI/z-stream-test-pipeline-nas/250220-063841/sequential_2025.02.20-06.39.59.log 🚀 File has been uploaded to CI/z-stream-test-pipeline-nas/250220-063841/parallel_2025.02.20-06.39.59.xml 🚀 File has been uploaded to CI/z-stream-test-pipeline-nas/250220-063841/parallel_2025.02.20-06.39.59.log 🚀 Updating artifacts path to results http://uploader.devtools-gitops--pipeline.svc.cluster.local:8080/CI/z-stream-test-pipeline-nas/250220-063841/1 undefined step-run-argocd-e2e-tests $RUN_ARGOCD_E2E_TESTS is set to false, skipping the execution 0 http://uploader.devtools-gitops--pipeline.svc.cluster.local:8080/CI/z-stream-test-pipeline-nas/250220-063841/ undefined step-evaluate-failed-tasks KuTTL test execution has failed with exit code 1 Visit http://uploader.devtools-gitops--pipeline.svc.cluster.local:8080/CI/z-stream-test-pipeline-nas/250220-063841/ to fetch the results or refer to the run-kuttl-tests task undefined step-send-slack-notification ok