undefined step-generate Cluster name generated: nasrosa-415-120454 Is nightly?: false Image repository: devtools_gitops Result path: CI/z-stream-test-pipeline/241204-052308 Result path URL: http://uploader.devtools-gitops--pipeline.svc.cluster.local:8080/CI/z-stream-test-pipeline/241204-052308/ undefined step-create-cluster-dir Directory with cluster info exists, logging in WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 104 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. undefined step-clone + '[' false = true ']' + '[' false = true ']' + '[' false = true ']' + CHECKOUT_DIR=/workspace/output/ + '[' false = true ']' + test -z '' + test -z '' + test -z '' + git config --global --add safe.directory /workspace/output + /ko-app/git-init -url=https://gitlab.cee.redhat.com/gitops/plumbing-gitops.git -revision= -refspec= -path=/workspace/output/ -sslVerify=false -submodules=true -depth=1 -sparseCheckoutDirectories= {"level":"info","ts":1733289799.992544,"caller":"git/git.go:176","msg":"Successfully cloned https://gitlab.cee.redhat.com/gitops/plumbing-gitops.git @ a4859e5c72de7caa0f0b3995cae8d9b545ba0ba7 (grafted, HEAD) in path /workspace/output/"} {"level":"info","ts":1733289800.0985246,"caller":"git/git.go:215","msg":"Successfully initialized and updated submodules in path /workspace/output/"} + cd /workspace/output/ ++ git rev-parse HEAD + RESULT_SHA=a4859e5c72de7caa0f0b3995cae8d9b545ba0ba7 + EXIT_CODE=0 + '[' 0 '!=' 0 ']' ++ git log -1 --pretty=%ct + RESULT_COMMITTER_DATE=1729838808 + printf %s 1729838808 + printf %s a4859e5c72de7caa0f0b3995cae8d9b545ba0ba7 + printf %s https://gitlab.cee.redhat.com/gitops/plumbing-gitops.git undefined step-create-catalog-source Logging in as cluster-admin user WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 104 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. NAME DISPLAY TYPE PUBLISHER AGE iib-872780 iib-872780-devtools_gitops grpc GitOps Team 107m pod/iib-872780-jc6ln condition met Operator will be installed from default catalog source iib-872780 passed undefined step-branch-version gitops version latest is Present undefined step-install-gitops-operator Logging in as cluster-admin user WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 104 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. Using iib-872780 catalog source to install the operator, subscribing to latest channel subscription.operators.coreos.com/openshift-gitops-operator created No resources found in openshift-operators namespace. No resources found in openshift-operators namespace. No resources found in openshift-operators namespace. pod/openshift-gitops-operator-controller-manager-6fbd46d959-rmmrv condition met namespace/openshift-gitops condition met deployment "cluster" successfully rolled out deployment "gitops-plugin" successfully rolled out deployment "openshift-gitops-applicationset-controller" successfully rolled out deployment "openshift-gitops-dex-server" successfully rolled out deployment "openshift-gitops-redis" successfully rolled out deployment "openshift-gitops-repo-server" successfully rolled out Waiting for deployment "openshift-gitops-server" rollout to finish: 0 of 1 updated replicas are available... deployment "openshift-gitops-server" successfully rolled out partitioned roll out complete: 1 new pods have been updated... argocd.argoproj.io/openshift-gitops condition met passed undefined step-test-argo-rollouts + RESULT=/install-dir/nasrosa-415-result.txt Logging in as cluster-admin user + trap update_result EXIT + echo -e 'Logging in as cluster-admin user' ++ cat /install-dir/nasrosa-415/auth/kubeadmin-password ++ cat /install-dir/nasrosa-415/auth/api-url + oc login -u cluster-admin -p Xy0oFb5poH1wslHpz https://api.nasrosa-415.ddru.p1.openshiftapps.com:6443 --insecure-skip-tls-verify=true WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 105 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. ++ echo true ++ tr '[:upper:]' '[:lower:]' + [[ true == \f\a\l\s\e ]] + exit_code=0 + failed=0 ++ oc get deployment openshift-gitops-operator-controller-manager -n openshift-gitops-operator -o 'jsonpath={.metadata.namespace}' --ignore-not-found + openshift_gitops_operator= + operator_namespace=openshift-operators + oc patch subscription openshift-gitops-operator -n openshift-operators --type merge --patch '{"spec": {"config": {"env": [{"name": "CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES", "value": "argo-rollouts,test-rom-ns-1,rom-ns-1"}]}}}' subscription.operators.coreos.com/openshift-gitops-operator patched ++ mktemp -d + ROLLOUTS_TMP_DIR=/tmp/tmp.ofaHpIuBVk + cd /tmp/tmp.ofaHpIuBVk + git clone https://github.com/argoproj-labs/argo-rollouts-manager Cloning into 'argo-rollouts-manager'... + cd /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager + TARGET_ROLLOUT_MANAGER_COMMIT=9f79ece2e923cbf03fe041bb6d1d83aae16a08da + TARGET_OPENSHIFT_ROUTE_ROLLOUT_PLUGIN_COMMIT=8b4125a7f9ecffb0247df91a4c890f88c0c523b7 + git checkout 9f79ece2e923cbf03fe041bb6d1d83aae16a08da Note: switching to '9f79ece2e923cbf03fe041bb6d1d83aae16a08da'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 9f79ece Fix typo in test comment + make test-e2e NAMESPACE_SCOPED_ARGO_ROLLOUTS=false hack/run-rollouts-manager-e2e-tests.sh + '[' false == true ']' + go test -v -p=1 -timeout=30m -race -count=1 -coverprofile=coverage.out ./tests/e2e/cluster-scoped go: downloading sigs.k8s.io/controller-runtime v0.16.3 go: downloading github.com/onsi/gomega v1.27.10 go: downloading github.com/onsi/ginkgo/v2 v2.11.0 go: downloading go.uber.org/zap v1.25.0 go: downloading k8s.io/api v0.28.3 go: downloading k8s.io/apimachinery v0.28.3 go: downloading k8s.io/client-go v0.28.3 go: downloading github.com/coreos/prometheus-operator v0.40.0 go: downloading k8s.io/apiextensions-apiserver v0.28.3 go: downloading github.com/go-logr/logr v1.2.4 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading go.uber.org/multierr v1.11.0 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/go-logr/zapr v1.2.4 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading k8s.io/klog/v2 v2.100.1 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading k8s.io/utils v0.0.0-20230406110748-d93618cff8a2 go: downloading golang.org/x/net v0.23.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd go: downloading k8s.io/kube-openapi v0.0.0-20230717233707-2695361300d9 go: downloading github.com/pkg/errors v0.9.1 go: downloading golang.org/x/sys v0.18.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading gomodules.xyz/jsonpatch/v2 v2.4.0 go: downloading github.com/golang/protobuf v1.5.3 go: downloading golang.org/x/time v0.3.0 go: downloading github.com/imdario/mergo v0.3.13 go: downloading github.com/spf13/pflag v1.0.5 go: downloading golang.org/x/term v0.18.0 go: downloading github.com/google/gnostic-models v0.6.8 go: downloading k8s.io/component-base v0.28.3 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading golang.org/x/exp v0.0.0-20220827204233-334a2380cb91 go: downloading github.com/prometheus/client_golang v1.16.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading golang.org/x/oauth2 v0.10.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading google.golang.org/protobuf v1.33.0 go: downloading github.com/prometheus/client_model v0.4.0 go: downloading github.com/prometheus/common v0.44.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.2.0 go: downloading github.com/prometheus/procfs v0.10.1 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.22.3 go: downloading github.com/go-openapi/jsonreference v0.20.2 go: downloading golang.org/x/text v0.14.0 go: downloading github.com/emicklei/go-restful/v3 v3.11.0 go: downloading github.com/go-openapi/jsonpointer v0.19.6 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4 go: downloading github.com/josharian/intern v1.0.0 === RUN TestClusterScoped Running Suite: Cluster-scoped Suite - /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager/tests/e2e/cluster-scoped ======================================================================================================== Random Seed: 1733290209 Will run 21 of 21 specs HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled True 0 2024-12-04 05:30:10 +0000 UTC Success } Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts-aggregate-to-admin Object exists in ExistByName: argo-rollouts-aggregate-to-edit Object exists in ExistByName: argo-rollouts-aggregate-to-view Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts-metrics Object exists in ExistByName: argo-rollouts-notification-secret Object exists in ExistByName: argo-rollouts Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled True 0 2024-12-04 05:31:00 +0000 UTC Success } Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts-aggregate-to-admin Object exists in ExistByName: argo-rollouts-aggregate-to-edit Object exists in ExistByName: argo-rollouts-aggregate-to-view Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts-metrics Object exists in ExistByName: argo-rollouts-notification-secret Object exists in ExistByName: argo-rollouts Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled True 0 2024-12-04 05:32:15 +0000 UTC Success } Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview HavePhase: expected: Failure actual: Failure HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerScope when Subscription has environment variable NAMESPACE_SCOPED_ARGO_ROLLOUTS set to False, there may not exist any namespace-scoped RolloutManagers: only a single cluster-scoped RolloutManager is supported} actual: {Reconciled False 0 2024-12-04 05:32:34 +0000 UTC InvalidRolloutManagerScope when Subscription has environment variable NAMESPACE_SCOPED_ARGO_ROLLOUTS set to False, there may not exist any namespace-scoped RolloutManagers: only a single cluster-scoped RolloutManager is supported} Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview HavePhase: expected: Available actual: Available HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled True 0 2024-12-04 05:32:15 +0000 UTC Success } Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled True 0 2024-12-04 05:33:25 +0000 UTC Success } Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview HavePhase: expected: Failure actual: Failure HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} actual: {Reconciled False 0 2024-12-04 05:33:42 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview HavePhase: expected: Failure actual: Failure HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} actual: {Reconciled False 0 2024-12-04 05:33:42 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} Object exists in ExistByName: rollout-bluegreen-active Object exists in ExistByName: rollout-bluegreen-preview •HavePhase: expected: Failure actual: Failure HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC InvalidRolloutManagerNamespace namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} actual: {Reconciled False 0 2024-12-04 05:34:33 +0000 UTC InvalidRolloutManagerNamespace Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES environment variable of Subscription resource. If you wish to install a cluster-scoped Argo Rollouts instance outside the default namespace, ensure it is defined in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES} ------------------------------ • [FAILED] [105.708 seconds] Cluster-scoped RolloutManager tests Testing cluster-scoped RolloutManager behaviour [It] Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES, then cluster-scoped RolloutManager should not be allowed. /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager/tests/e2e/cluster-scoped/cluster_scoped_rollouts_test.go:289 Timeline >> STEP: Create namespace. @ 12/04/24 05:34:33.054 STEP: Verify that RolloutManager is successfully created. @ 12/04/24 05:34:33.15 STEP: Verify that Status.Condition is now having error message. @ 12/04/24 05:34:33.188 [FAILED] in [It] - /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager/tests/e2e/cluster-scoped/cluster_scoped_rollouts_test.go:302 @ 12/04/24 05:35:33.189 << Timeline [FAILED] Timed out after 60.001s. Expected : false to be true In [It] at: /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager/tests/e2e/cluster-scoped/cluster_scoped_rollouts_test.go:302 @ 12/04/24 05:35:33.189 ------------------------------ HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled True 0 2024-12-04 05:35:53 +0000 UTC Success } HavePhase: expected: Failure actual: Failure HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} actual: {Reconciled False 0 2024-12-04 05:36:09 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} HaveCondition: expected: {Reconciled False 0 0001-01-01 00:00:00 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} actual: {Reconciled False 0 2024-12-04 05:36:09 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled False 0 2024-12-04 05:36:09 +0000 UTC MultipleClusterScopedRolloutManager when there exists a cluster-scoped RolloutManager on the cluster, there may not exist another: only a single cluster-scoped RolloutManager is supported} HaveSuccessCondition: expected: {Reconciled True 0 0001-01-01 00:00:00 +0000 UTC Success } actual: {Reconciled True 0 2024-12-04 05:36:09 +0000 UTC Success } •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object does not exists in ExistByName: argo-rollouts-aggregate-to-admin clusterroles.rbac.authorization.k8s.io "argo-rollouts-aggregate-to-admin" not found Object exists in ExistByName: argo-rollouts-aggregate-to-view Object does not exists in ExistByName: argo-rollouts-aggregate-to-view clusterroles.rbac.authorization.k8s.io "argo-rollouts-aggregate-to-view" not found Object does not exists in ExistByName: argo-rollouts-aggregate-to-edit clusterroles.rbac.authorization.k8s.io "argo-rollouts-aggregate-to-edit" not found •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts-aggregate-to-admin Object exists in ExistByName: argo-rollouts-aggregate-to-edit Object exists in ExistByName: argo-rollouts-aggregate-to-view Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts-metrics Object exists in ExistByName: argo-rollouts-notification-secret Object exists in ExistByName: argo-rollouts •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object does not exists in ExistByName: argo-rollouts serviceaccounts "argo-rollouts" not found Object does not exists in ExistByName: argo-rollouts clusterroles.rbac.authorization.k8s.io "argo-rollouts" not found Object does not exists in ExistByName: argo-rollouts clusterrolebindings.rbac.authorization.k8s.io "argo-rollouts" not found Object does not exists in ExistByName: argo-rollouts-aggregate-to-admin clusterroles.rbac.authorization.k8s.io "argo-rollouts-aggregate-to-admin" not found Object does not exists in ExistByName: argo-rollouts-aggregate-to-edit clusterroles.rbac.authorization.k8s.io "argo-rollouts-aggregate-to-edit" not found Object does not exists in ExistByName: argo-rollouts-aggregate-to-view clusterroles.rbac.authorization.k8s.io "argo-rollouts-aggregate-to-view" not found Object does not exists in ExistByName: argo-rollouts deployments.apps "argo-rollouts" not found Object does not exists in ExistByName: argo-rollouts-metrics services "argo-rollouts-metrics" not found Object does not exists in ExistByName: argo-rollouts-notification-secret secrets "argo-rollouts-notification-secret" not found Object does not exists in ExistByName: argo-rollouts servicemonitors.monitoring.coreos.com "argo-rollouts" not found •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object exists in ExistByName: argo-rollouts •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object exists in ExistByName: argo-rollouts •HavePhase: expected: Pending actual: HavePhase: expected: Pending actual: Pending Object exists in ExistByName: argo-rollouts Object exists in ExistByName: argo-rollouts •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object does not exists in ExistByName: argo-rollouts-notification-secret secrets "argo-rollouts-notification-secret" not found Object exists in ExistByName: argo-rollouts-notification-secret •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object exists in ExistByName: argo-rollouts-notification-secret Object does not exists in ExistByName: argo-rollouts-notification-secret secrets "argo-rollouts-notification-secret" not found •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available Object exists in ExistByName: argo-rollouts-notification-secret Object exists in ExistByName: argo-rollouts-notification-secret •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available HavePhase: expected: Available actual: Available •HavePhase: expected: Available actual: HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Pending HavePhase: expected: Available actual: Available •Object does not exists in ExistByName: argo-rollouts deployments.apps "argo-rollouts" not found Object exists in ExistByName: argo-rollouts • Summarizing 1 Failure: [FAIL] Cluster-scoped RolloutManager tests Testing cluster-scoped RolloutManager behaviour [It] Namespace is not specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES, then cluster-scoped RolloutManager should not be allowed. /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager/tests/e2e/cluster-scoped/cluster_scoped_rollouts_test.go:302 Ran 21 of 21 Specs in 818.904 seconds FAIL! -- 20 Passed | 1 Failed | 0 Pending | 0 Skipped --- FAIL: TestClusterScoped (818.91s) FAIL coverage: [no statements] FAIL github.com/argoproj-labs/argo-rollouts-manager/tests/e2e/cluster-scoped 819.011s FAIL make: *** [Makefile:120: test-e2e] Error 1 + exit_code=2 + [[ 2 != 0 ]] + echo 2 + tee /tekton/results/result 2 + failed=2 + exit_code=0 + kubectl delete rolloutmanagers --all -n test-rom-ns-1 No resources found + cd /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager + SKIP_RUN_STEP=true + hack/run-upstream-argo-rollouts-e2e-tests.sh + set -e + trap cleanup EXIT +++ dirname -- hack/run-upstream-argo-rollouts-e2e-tests.sh ++ cd -- hack ++ pwd + SCRIPT_DIR=/tmp/tmp.ofaHpIuBVk/argo-rollouts-manager/hack ++ mktemp -d + TMP_DIR=/tmp/tmp.ghLtMCCrSg + cd /tmp/tmp.ghLtMCCrSg + git clone https://github.com/argoproj/argo-rollouts Cloning into 'argo-rollouts'... + cd argo-rollouts + git checkout v1.7.2 Note: switching to 'v1.7.2'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 59e5bd385 fix: replicaSet not scaled down due to incorrect annotations (#3762) (#3784) + go mod tidy go: downloading github.com/argoproj/pkg v0.13.6 go: downloading github.com/sirupsen/logrus v1.9.3 go: downloading github.com/servicemeshinterface/smi-sdk-go v0.5.0 go: downloading github.com/spf13/cobra v1.8.0 go: downloading k8s.io/apimachinery v0.29.3 go: downloading k8s.io/client-go v0.29.3 go: downloading k8s.io/cli-runtime v0.29.3 go: downloading github.com/blang/semver v3.5.1+incompatible go: downloading k8s.io/klog/v2 v2.110.1 go: downloading k8s.io/apiextensions-apiserver v0.29.3 go: downloading k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 go: downloading github.com/grpc-ecosystem/grpc-gateway v1.16.0 go: downloading sigs.k8s.io/yaml v1.4.0 go: downloading k8s.io/code-generator v0.29.3 go: downloading github.com/aws/aws-sdk-go-v2/config v1.27.10 go: downloading github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.37.0 go: downloading github.com/aws/aws-sdk-go-v2 v1.26.1 go: downloading github.com/stretchr/testify v1.9.0 go: downloading k8s.io/utils v0.0.0-20230726121419-3b25d923346b go: downloading github.com/argoproj/notifications-engine v0.4.1-0.20240219110818-7a069766e954 go: downloading k8s.io/api v0.29.3 go: downloading github.com/aws/smithy-go v1.20.2 go: downloading github.com/hashicorp/go-plugin v1.6.0 go: downloading github.com/machinebox/graphql v0.2.2 go: downloading github.com/tj/assert v0.0.3 go: downloading k8s.io/kubernetes v1.29.3 go: downloading github.com/spaceapegames/go-wavefront v1.8.1 go: downloading github.com/golang/protobuf v1.5.4 go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20240123012728-ef4313101c80 go: downloading github.com/newrelic/newrelic-client-go v1.1.0 go: downloading google.golang.org/grpc v1.62.1 go: downloading golang.org/x/oauth2 v0.18.0 go: downloading google.golang.org/genproto v0.0.0-20240123012728-ef4313101c80 go: downloading github.com/prometheus/client_golang v1.18.0 go: downloading k8s.io/component-base v0.29.3 go: downloading github.com/prometheus/common v0.47.0 go: downloading github.com/prometheus/common/sigv4 v0.1.0 go: downloading github.com/juju/ansiterm v1.0.0 go: downloading k8s.io/kubectl v0.29.3 go: downloading github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 v1.30.5 go: downloading k8s.io/apiserver v0.29.3 go: downloading github.com/soheilhy/cmux v0.1.5 go: downloading github.com/influxdata/influxdb-client-go/v2 v2.13.0 go: downloading github.com/antonmedv/expr v1.15.5 go: downloading github.com/evanphx/json-patch/v5 v5.9.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/bombsimon/logrusr/v4 v4.1.0 go: downloading github.com/valyala/fasttemplate v1.2.2 go: downloading github.com/golang/mock v1.6.0 go: downloading github.com/prometheus/client_model v0.6.0 go: downloading github.com/inconshreveable/mousetrap v1.1.0 go: downloading github.com/go-logr/logr v1.3.0 go: downloading github.com/google/uuid v1.6.0 go: downloading github.com/golang/glog v1.2.0 go: downloading github.com/google/go-cmp v0.6.0 go: downloading k8s.io/gengo v0.0.0-20230829151522-9cce18d56c01 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.17.10 go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.5 go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.1 go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.20.4 go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.23.4 go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.28.6 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/stretchr/objx v0.5.2 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.4.1 go: downloading github.com/felixge/httpsnoop v1.0.3 go: downloading golang.org/x/net v0.22.0 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20240123012728-ef4313101c80 go: downloading github.com/matryer/is v1.4.0 go: downloading github.com/hashicorp/go-hclog v0.14.1 go: downloading github.com/hashicorp/yamux v0.1.1 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/prometheus/procfs v0.12.0 go: downloading github.com/mattn/go-colorable v0.1.13 go: downloading github.com/mattn/go-isatty v0.0.19 go: downloading github.com/aws/aws-sdk-go v1.44.116 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/jhump/protoreflect v1.15.1 go: downloading github.com/elazarl/goproxy v0.0.0-20220417044921-416226498f94 go: downloading github.com/influxdata/line-protocol v0.0.0-20210922203350-b1ad95c89adf go: downloading github.com/oapi-codegen/runtime v1.0.0 go: downloading github.com/PagerDuty/go-pagerduty v1.7.0 go: downloading github.com/RocketChat/Rocket.Chat.Go.SDK v0.0.0-20220708192748-b73dcb041214 go: downloading github.com/aws/aws-sdk-go-v2/service/sqs v1.29.7 go: downloading github.com/bradleyfalzon/ghinstallation/v2 v2.5.0 go: downloading github.com/OvyFlash/telegram-bot-api/v5 v5.0.0-20240108230938-63e5c59035bf go: downloading github.com/google/go-github/v41 v41.0.0 go: downloading github.com/gregdel/pushover v1.2.1 go: downloading github.com/hashicorp/go-retryablehttp v0.7.1 go: downloading github.com/opsgenie/opsgenie-go-sdk-v2 v1.2.13 go: downloading github.com/slack-go/slack v0.12.2 go: downloading github.com/spf13/cast v1.5.1 go: downloading github.com/whilp/git-urls v0.0.0-20191001220047-6db9661140c0 go: downloading gomodules.xyz/notify v0.1.1 go: downloading google.golang.org/api v0.149.0 go: downloading github.com/valyala/bytebufferpool v1.0.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading github.com/moby/term v0.0.0-20221205130635-1aeaba878587 go: downloading golang.org/x/sync v0.6.0 go: downloading sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3 go: downloading sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32 go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.5 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.7 go: downloading github.com/Masterminds/sprig/v3 v3.2.3 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading google.golang.org/appengine v1.6.8 go: downloading github.com/fatih/color v1.7.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/tomnomnom/linkheader v0.0.0-20180905144013-02ca5825eb80 go: downloading github.com/valyala/fastjson v1.6.3 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f go: downloading github.com/lunixbochs/vtclean v1.0.0 go: downloading github.com/kr/pretty v0.3.1 go: downloading github.com/apapsch/go-jsonmerge/v2 v2.0.0 go: downloading github.com/onsi/ginkgo/v2 v2.13.0 go: downloading github.com/onsi/gomega v1.29.0 go: downloading github.com/golang-jwt/jwt/v4 v4.5.0 go: downloading github.com/google/go-github/v53 v53.0.0 go: downloading golang.org/x/crypto v0.21.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/gorilla/websocket v1.5.0 go: downloading github.com/go-test/deep v1.0.4 go: downloading github.com/frankban/quicktest v1.14.4 go: downloading gomodules.xyz/envconfig v1.3.1-0.20190308184047-426f31af0d45 go: downloading gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df go: downloading github.com/exponent-io/jsonpath v0.0.0-20210407135951-1de76d718b3f go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.1 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading k8s.io/component-helpers v0.29.3 go: downloading k8s.io/kubelet v0.29.3 go: downloading github.com/creack/pty v1.1.18 go: downloading golang.org/x/tools v0.16.1 go: downloading github.com/google/btree v1.1.2 go: downloading github.com/bufbuild/protocompile v0.4.0 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.2.0 go: downloading github.com/huandu/xstrings v1.3.3 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/jpillora/backoff v1.0.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.10.0 go: downloading github.com/Jeffail/gabs v1.4.0 go: downloading github.com/gopackage/ddp v0.0.0-20170117053602-652027933df4 go: downloading github.com/onsi/ginkgo v1.14.1 go: downloading github.com/sony/sonyflake v1.0.0 go: downloading gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc go: downloading github.com/distribution/reference v0.5.0 go: downloading k8s.io/cloud-provider v0.29.3 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/ProtonMail/go-crypto v0.0.0-20230217124315-7d5c6f04bbb8 go: downloading k8s.io/cluster-bootstrap v0.25.8 go: downloading github.com/go-errors/errors v1.4.2 go: downloading github.com/sergi/go-diff v1.1.0 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.2.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f go: downloading go.starlark.net v0.0.0-20230525235612-a134d8f9ddca go: downloading k8s.io/controller-manager v0.29.3 go: downloading github.com/fsnotify/fsnotify v1.7.0 go: downloading github.com/coreos/go-systemd/v22 v22.5.0 go: downloading go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.19.0 go: downloading go.opentelemetry.io/otel/sdk v1.19.0 go: downloading go.opentelemetry.io/otel v1.19.0 go: downloading gopkg.in/natefinch/lumberjack.v2 v2.2.1 go: downloading github.com/google/cel-go v0.17.7 go: downloading go.opentelemetry.io/otel/trace v1.19.0 go: downloading go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.19.0 go: downloading go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.44.0 go: downloading sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.28.0 go: downloading k8s.io/kms v0.29.3 go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 go: downloading go.etcd.io/etcd/client/pkg/v3 v3.5.10 go: downloading go.etcd.io/etcd/client/v3 v3.5.10 go: downloading go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.42.0 go: downloading go.uber.org/zap v1.19.0 go: downloading github.com/NYTimes/gziphandler v1.1.1 go: downloading github.com/cloudflare/circl v1.3.3 go: downloading github.com/stoewer/go-strcase v1.2.0 go: downloading go.opentelemetry.io/otel/metric v1.19.0 go: downloading go.uber.org/goleak v1.2.1 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.etcd.io/etcd/api/v3 v3.5.10 go: downloading go.etcd.io/etcd/server/v3 v3.5.10 go: downloading go.opentelemetry.io/proto/otlp v1.0.0 go: downloading github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 go: downloading github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a go: downloading github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20230305170008-8188dc5388df go: downloading github.com/go-logr/stdr v1.2.2 go: downloading github.com/cenkalti/backoff/v4 v4.2.1 go: downloading go.uber.org/atomic v1.10.0 go: downloading github.com/benbjohnson/clock v1.1.0 go: downloading github.com/cncf/xds/go v0.0.0-20231128003011-0fa0005c9caa go: downloading github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 go: downloading github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 go: downloading github.com/coreos/go-semver v0.3.1 go: downloading golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e go: downloading cloud.google.com/go/compute/metadata v0.2.3 go: downloading github.com/envoyproxy/protoc-gen-validate v1.0.4 go: downloading cloud.google.com/go/compute v1.23.3 go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20220101234140-673ab2c3ae75 go: downloading go.etcd.io/bbolt v1.3.8 go: downloading go.etcd.io/etcd/pkg/v3 v3.5.10 go: downloading go.etcd.io/etcd/raft/v3 v3.5.10 go: downloading github.com/dustin/go-humanize v1.0.1 go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 go: downloading github.com/jonboulle/clockwork v0.2.2 go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 go: downloading golang.org/x/mod v0.14.0 go: downloading go.etcd.io/etcd/client/v2 v2.305.10 go: downloading go.opencensus.io v0.24.0 go: downloading github.com/google/s2a-go v0.1.7 go: downloading github.com/googleapis/gax-go/v2 v2.12.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.3.2 + find /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e -type f + xargs sed -i.bak 's/argoproj\/rollouts-demo/quay.io\/jgwest-redhat\/rollouts-demo/g' + kubectl delete ns argo-rollouts namespace "argo-rollouts" deleted + kubectl wait --timeout=5m --for=delete namespace/argo-rollouts + kubectl create ns argo-rollouts namespace/argo-rollouts created + kubectl config set-context --current --namespace=argo-rollouts Context "default/api-nasrosa-415-ddru-p1-openshiftapps-com:6443/cluster-admin" modified. + cd /tmp/tmp.ofaHpIuBVk/argo-rollouts-manager/hack/.. + '[' -z true ']' + cd /tmp/tmp.ghLtMCCrSg/argo-rollouts + cat + kubectl apply -f /tmp/tmp.ghLtMCCrSg/rollout-manager.yaml rolloutmanager.argoproj.io/argo-rollout created + echo '* Waiting for Argo Rollouts Deployment to exist' * Waiting for Argo Rollouts Deployment to exist + kubectl get -n argo-rollouts deployment/argo-rollouts NAME READY UP-TO-DATE AVAILABLE AGE argo-rollouts 0/1 1 0 0s + kubectl wait --for=condition=Available --timeout=10m -n argo-rollouts deployment/argo-rollouts deployment.apps/argo-rollouts condition met + kubectl apply -f test/e2e/crds customresourcedefinition.apiextensions.k8s.io/apisixroutes.apisix.apache.org configured customresourcedefinition.apiextensions.k8s.io/wasmplugins.extensions.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/destinationrules.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/envoyfilters.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/gateways.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/proxyconfigs.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/serviceentries.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/sidecars.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/virtualservices.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/workloadentries.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/workloadgroups.networking.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/authorizationpolicies.security.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/peerauthentications.security.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/requestauthentications.security.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/telemetries.telemetry.istio.io unchanged customresourcedefinition.apiextensions.k8s.io/trafficsplits.split.smi-spec.io unchanged + oc adm policy add-scc-to-user anyuid -z argo-rollouts -n argo-rollouts clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "argo-rollouts" + oc adm policy add-scc-to-user anyuid -z default -n argo-rollouts clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "default" + rm -f /tmp/test-e2e.log + set +e + make test-e2e + tee /tmp/test-e2e.log ./hack/installers/install-dev-tools.sh +++ dirname ./hack/installers/install-dev-tools.sh ++ cd ./hack/installers/../.. ++ pwd + PROJECT_ROOT=/tmp/tmp.ghLtMCCrSg/argo-rollouts + DIST_PATH=/tmp/tmp.ghLtMCCrSg/argo-rollouts/dist + PATH=/tmp/tmp.ghLtMCCrSg/argo-rollouts/dist:/tmp/tmp.ghLtMCCrSg/argo-rollouts/dist:/root/.nvm/versions/node/v20.12.2/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin + mkdir -p /tmp/tmp.ghLtMCCrSg/argo-rollouts/dist + gotestsum_version=1.10.1 ++ go env GOOS + OS=linux ++ go env GOARCH + ARCH=amd64 + export TARGET_FILE=gotestsum_1.10.1_linux_amd64.tar.gz + TARGET_FILE=gotestsum_1.10.1_linux_amd64.tar.gz + temp_path=/tmp/gotestsum_1.10.1_linux_amd64.tar.gz + url=https://github.com/gotestyourself/gotestsum/releases/download/v1.10.1/gotestsum_1.10.1_linux_amd64.tar.gz + '[' -e /tmp/gotestsum_1.10.1_linux_amd64.tar.gz ']' + curl -sLf --retry 3 -o /tmp/gotestsum_1.10.1_linux_amd64.tar.gz https://github.com/gotestyourself/gotestsum/releases/download/v1.10.1/gotestsum_1.10.1_linux_amd64.tar.gz + mkdir -p /tmp/gotestsum-1.10.1 + tar -xvzf /tmp/gotestsum_1.10.1_linux_amd64.tar.gz -C /tmp/gotestsum-1.10.1 LICENSE LICENSE.md README.md gotestsum + cp /tmp/gotestsum-1.10.1/gotestsum /tmp/tmp.ghLtMCCrSg/argo-rollouts/dist/gotestsum + chmod +x /tmp/tmp.ghLtMCCrSg/argo-rollouts/dist/gotestsum + gotestsum --version gotestsum version 1.10.1 /tmp/tmp.ghLtMCCrSg/argo-rollouts/dist/gotestsum --rerun-fails-report=rerunreport.txt --junitfile=junit.xml --format=testname --packages="./test/e2e" --rerun-fails=5 -- -timeout 60m -count 1 --tags e2e -p 1 -parallel 1 -v --short ./test/e2e PASS test/e2e.TestAnalysisSuite/TestAnalysisWithArgs (39.95s) PASS test/e2e.TestAnalysisSuite/TestAnalysisWithSecret (29.81s) PASS test/e2e.TestAnalysisSuite/TestBackgroundAnalysisWithArgs (28.46s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenAbortAndUpdate (21.17s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenAnalysis (28.79s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenKitchenSink (44.89s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenPostPromotionFail (19.50s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenPrePromotionFail (29.97s) PASS test/e2e.TestAnalysisSuite/TestCanaryBackgroundAnalysis (4.33s) PASS test/e2e.TestAnalysisSuite/TestCanaryFailInlineMultipleAnalysis (11.66s) PASS test/e2e.TestAnalysisSuite/TestCanaryInlineAnalysis (16.07s) PASS test/e2e.TestAnalysisSuite/TestCanaryInlineMultipleAnalysis (18.05s) PASS test/e2e.TestAnalysisSuite/TestMultipleAnalysis (6.19s) PASS test/e2e.TestAnalysisSuite (306.93s) === RUN TestAPISIXSuite/TestAPISIXCanarySetHeaderStep time="2024-12-04T06:02:40Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:02:41Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:42Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:43Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:45Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:45Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:02:45Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.36s) FAIL test/e2e.TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.36s) PASS test/e2e.TestAPISIXSuite/TestAPISIXCanaryStep (26.13s) === RUN TestAPISIXSuite --- FAIL: TestAPISIXSuite (31.90s) FAIL test/e2e.TestAPISIXSuite (31.90s) PASS test/e2e.TestAWSSuite/TestALBExperimentStep (23.86s) PASS test/e2e.TestAWSSuite/TestALBExperimentStepMultiIngress (25.14s) PASS test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeight (24.14s) === RUN TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress time="2024-12-04T06:04:27Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" time="2024-12-04T06:04:29Z" level=info msg="service/alb-rollout-root created\n" rollout=alb-rollout time="2024-12-04T06:04:30Z" level=info msg="service/alb-rollout-canary created\n" rollout=alb-rollout time="2024-12-04T06:04:32Z" level=info msg="service/alb-rollout-stable created\n" rollout=alb-rollout time="2024-12-04T06:04:34Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-1 created\n" rollout=alb-rollout time="2024-12-04T06:04:35Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-2 created\n" rollout=alb-rollout time="2024-12-04T06:04:37Z" level=info msg="rollout.argoproj.io/alb-rollout created\n" rollout=alb-rollout time="2024-12-04T06:04:37Z" level=info msg="Waiting for condition: status=Healthy" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Condition 'status=Healthy' met after 1s" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Experiment expectation 'experiment count == 0' met" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Updated rollout pod spec: 2024-12-04T06:04:38.668123905Z" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Sleeping 10s" rollout=alb-rollout require.go:72: Error Trace: /root/go/pkg/mod/github.com/tj/assert@v0.0.3/require.go:72 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:365 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/fixtures/then.go:29 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:354 Error: Not equal: expected: "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" actual : "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" Diff: --- Expected +++ Actual @@ -1 +1 @@ -{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} +{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServicePort":"80","Weight":20},{"ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} Test: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress Name: alb-rollout Namespace: argo-rollouts Status: ◌ Progressing Message: more replicas need to be updated Strategy: Canary Step: 0/1 SetWeight: 0 ActualWeight: 0 Images: nginx:1.19-alpine (stable, Σ:experiment-alb-canary, Σ:experiment-alb-stable) Replicas: Desired: 1 Current: 1 Updated: 0 Ready: 1 Available: 1 NAME KIND STATUS AGE INFO ⟳ alb-rollout Rollout ◌ Progressing 11s ├──# revision:2 │ ├──⧉ alb-rollout-84fd759574 ReplicaSet • ScaledDown 10s canary │ └──Σ alb-rollout-84fd759574-2-0 Experiment ◌ Running 10s │ ├──⧉ alb-rollout-84fd759574-2-0-experiment-alb-canary ReplicaSet ✔ Healthy 10s │ │ └──□ alb-rollout-84fd759574-2-0-experiment-alb-canary-kgj7q Pod ✔ Running 10s ready:1/1 │ └──⧉ alb-rollout-84fd759574-2-0-experiment-alb-stable ReplicaSet ✔ Healthy 10s │ └──□ alb-rollout-84fd759574-2-0-experiment-alb-stable-qgcrs Pod ✔ Running 10s ready:1/1 └──# revision:1 └──⧉ alb-rollout-d6f4fdcdc ReplicaSet ✔ Healthy 11s stable └──□ alb-rollout-d6f4fdcdc-v57jm Pod ✔ Running 11s ready:1/1 --- apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: annotations: rollout.argoproj.io/revision: "2" creationTimestamp: "2024-12-04T06:04:37Z" generation: 3 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestALBExperimentStepNoSetWeightMultiIngress name: alb-rollout namespace: argo-rollouts resourceVersion: "2965663" uid: 4e5112df-b5a8-481b-b707-222bedc53d92 spec: replicas: 1 selector: matchLabels: app: alb-rollout strategy: canary: canaryService: alb-rollout-canary stableService: alb-rollout-stable steps: - experiment: analysisRunMetadata: {} duration: 15s templates: - metadata: {} name: experiment-alb-canary specRef: canary weight: 20 - metadata: {} name: experiment-alb-stable specRef: stable weight: 20 trafficRouting: alb: ingresses: - alb-rollout-multi-ingress-1 - alb-rollout-multi-ingress-2 rootService: alb-rollout-root servicePort: 80 template: metadata: annotations: update: "2024-12-04T06:04:38.668123905Z" creationTimestamp: null labels: app: alb-rollout spec: containers: - image: nginx:1.19-alpine name: alb-rollout ports: - containerPort: 80 name: http protocol: TCP resources: requests: cpu: 5m memory: 16Mi status: HPAReplicas: 1 availableReplicas: 1 blueGreen: {} canary: currentExperiment: alb-rollout-84fd759574-2-0 weights: additional: - weight: 20 - weight: 20 canary: podTemplateHash: 84fd759574 serviceName: alb-rollout-canary weight: 0 stable: podTemplateHash: d6f4fdcdc serviceName: alb-rollout-stable weight: 60 conditions: - lastTransitionTime: "2024-12-04T06:04:38Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: Rollout has minimum availability reason: AvailableReason status: "True" type: Available - lastTransitionTime: "2024-12-04T06:04:38Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: Rollout is not healthy reason: RolloutHealthy status: "False" type: Healthy - lastTransitionTime: "2024-12-04T06:04:38Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: RolloutCompleted reason: RolloutCompleted status: "False" type: Completed - lastTransitionTime: "2024-12-04T06:04:37Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: Created new replica set "alb-rollout-84fd759574" reason: NewReplicaSetCreated status: "True" type: Progressing currentPodHash: 84fd759574 currentStepHash: 8b59978 currentStepIndex: 0 message: more replicas need to be updated observedGeneration: "3" phase: Progressing readyReplicas: 1 replicas: 1 selector: app=alb-rollout stableRS: d6f4fdcdc 2024-12-04T06:04:37Z Normal RolloutAddedToInformer Rollout resource added to informer: argo-rollouts/alb-rollout 2024-12-04T06:04:37Z Normal RolloutNotCompleted Rollout not completed, started update to revision 1 (d6f4fdcdc) 2024-12-04T06:04:37Z Normal RolloutUpdated Rollout updated to revision 1 2024-12-04T06:04:37Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-d6f4fdcdc (revision 1) 2024-12-04T06:04:39Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-1` to desiredWeight '0' 2024-12-04T06:04:39Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-2` to desiredWeight '0' 2024-12-04T06:04:37Z Normal TrafficWeightUpdated Traffic weight updated to 0 2024-12-04T06:04:37Z Normal ScalingReplicaSet Scaled up ReplicaSet alb-rollout-d6f4fdcdc (revision 1) from 0 to 1 2024-12-04T06:04:37Z Normal RolloutCompleted Rollout completed update to revision 1 (d6f4fdcdc): Initial deploy 2024-12-04T06:04:38Z (x2) Normal TrafficWeightUpdated Traffic weight updated 2024-12-04T06:04:38Z Normal SwitchService Switched selector for service 'alb-rollout-stable' from '' to 'd6f4fdcdc' 2024-12-04T06:04:38Z Normal SwitchService Switched selector for service 'alb-rollout-canary' from '' to 'd6f4fdcdc' 2024-12-04T06:04:38Z Normal RolloutNotCompleted Rollout not completed, started update to revision 2 (84fd759574) 2024-12-04T06:04:38Z Normal RolloutUpdated Rollout updated to revision 2 2024-12-04T06:04:38Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-84fd759574 (revision 2) 2024-12-04T06:04:38Z Normal ExperimentCreated Created Experiment 'alb-rollout-84fd759574-2-0' 2024-12-04T06:04:39Z Normal TrafficWeightUpdated Traffic weight updated additional: [{20 } {20 }] time="2024-12-04T06:04:48Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" --- FAIL: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (22.12s) FAIL test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (22.12s) PASS test/e2e.TestAWSSuite/TestALBMesh_PingPong_StableCanary_Update (19.70s) PASS test/e2e.TestAWSSuite/TestALBPingPongUpdate (15.94s) PASS test/e2e.TestAWSSuite/TestALBPingPongUpdateMultiIngress (17.02s) PASS test/e2e.TestAWSSuite/TestAlbHeaderRoute (10.54s) PASS test/e2e.TestAWSSuite/TestAlbHeaderRouteMultiIngress (24.46s) === RUN TestAWSSuite --- FAIL: TestAWSSuite (185.42s) FAIL test/e2e.TestAWSSuite (185.42s) PASS test/e2e.TestBlueGreenSuite/TestBlueGreenPreviewReplicaCount (8.12s) PASS test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithPause (64.26s) PASS test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithoutPause (64.13s) PASS test/e2e.TestBlueGreenSuite/TestEphemeralMetadata (9.91s) PASS test/e2e.TestBlueGreenSuite (146.82s) PASS test/e2e.TestCanarySuite/TestCanaryDynamicStableScale (25.96s) PASS test/e2e.TestCanarySuite/TestCanaryDynamicStableScaleRollbackToStable (18.01s) PASS test/e2e.TestCanarySuite/TestCanaryProgressDeadlineExceededWithPause (124.16s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownDelay (13.76s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownDelayWithProgressDeadline (18.86s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbort (12.17s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbortNoTrafficRouting (11.03s) PASS test/e2e.TestCanarySuite/TestCanaryUnScaleDownOnAbort (14.17s) PASS test/e2e.TestCanarySuite/TestCanaryWithPausedRollout (102.84s) PASS test/e2e.TestCanarySuite/TestEphemeralMetadata (7.75s) PASS test/e2e.TestCanarySuite/TestReduceWeightAndHonorMaxUnavailable (30.01s) PASS test/e2e.TestCanarySuite/TestRolloutScalingDuringUpdate (3.71s) PASS test/e2e.TestCanarySuite/TestRolloutScalingWhenPaused (5.79s) PASS test/e2e.TestCanarySuite/TestRolloutWithMaxSurgeScalingDuringUpdate (3.92s) PASS test/e2e.TestCanarySuite/TestCanarySetCanaryScale (10.90s) PASS test/e2e.TestCanarySuite (394.08s) === RUN TestExperimentSuite/TestExperimentWithDryRunMetrics time="2024-12-04T06:15:28Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:15:30Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:15:32Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:15:32Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:16:03Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:16:03Z" level=info msg="Sleeping 3s" time="2024-12-04T06:16:06Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:16:06Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:16:06Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-c4hmv Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 96cdda97-79ea-4423-ac31-3b683c4a4577.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:15:32Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "2980655" uid: 3a90e476-6753-4fdf-ad21-e6e6f89e958d spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:15:33Z" conditions: - lastTransitionTime: "2024-12-04T06:16:03Z" lastUpdateTime: "2024-12-04T06:16:03Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:16:03Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:15:32Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:15:32Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:15:32Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:15:33Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:15:33Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:15:33Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:15:33Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:16:03Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:16:03Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:16:03Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:16:06Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (37.66s) FAIL test/e2e.TestExperimentSuite/TestExperimentWithDryRunMetrics (37.66s) PASS test/e2e.TestExperimentSuite/TestExperimentWithMeasurementRetentionMetrics (36.82s) PASS test/e2e.TestExperimentSuite/TestExperimentWithMultiportServiceAndScaleDownDelay (18.09s) PASS test/e2e.TestExperimentSuite/TestExperimentWithServiceAndScaleDownDelay (17.97s) PASS test/e2e.TestExperimentSuite/TestExperimentWithServiceNameAndScaleDownDelay (17.99s) PASS test/e2e.TestExperimentSuite/TestRolloutWithExperimentAndAnalysis (10.14s) === RUN TestExperimentSuite --- FAIL: TestExperimentSuite (129.31s) FAIL test/e2e.TestExperimentSuite (129.31s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenExceedProgressDeadlineAbort (10.24s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenPreviewReplicaCountPromoteFull (8.46s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenPromoteFull (9.44s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownDelay (9.95s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownOnAbort (9.87s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenToCanary (4.93s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenUpdate (6.25s) PASS test/e2e.TestFunctionalSuite/TestCanaryPromoteFull (9.67s) PASS test/e2e.TestFunctionalSuite/TestCompleteRolloutRestart (22.04s) PASS test/e2e.TestFunctionalSuite/TestContainerResourceFormats (3.08s) === RUN TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:19:23Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:19:23Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 6322 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc002458340, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc000bd1200) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000660d00?, 0xc0010165a0?, 0xc001239690?}, {0x3c1144c, 0x4}, {0xc0019b3e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000660d00?, 0xc0010165a0?, 0xc000bd1200?}, {0xc0019b3e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc002458340) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc002458340, 0xc000c8cea0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 5379 /usr/local/go/src/testing/testing.go:1648 +0x3ad --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.73s) FAIL test/e2e.TestFunctionalSuite/TestControllerMetrics (0.73s) PASS test/e2e.TestFunctionalSuite/TestFixInvalidSpec (22.03s) PASS test/e2e.TestFunctionalSuite/TestKubectlWaitForCompleted (24.18s) PASS test/e2e.TestFunctionalSuite/TestKubectlWaitForPaused (7.75s) PASS test/e2e.TestFunctionalSuite/TestMalformedRollout (6.41s) PASS test/e2e.TestFunctionalSuite/TestMalformedRolloutEphemeralCtr (3.02s) PASS test/e2e.TestFunctionalSuite/TestNeverScaleDown (6.33s) PASS test/e2e.TestFunctionalSuite/TestRolloutAbortRetryPromote (8.79s) PASS test/e2e.TestFunctionalSuite/TestRolloutBadToBadUpdate (23.95s) PASS test/e2e.TestFunctionalSuite/TestRolloutBadToGoodSetWeightUpdate (22.18s) PASS test/e2e.TestFunctionalSuite/TestRolloutBadToGoodUpdate (17.05s) PASS test/e2e.TestFunctionalSuite/TestRolloutGoodToBadUpdate (24.80s) PASS test/e2e.TestFunctionalSuite/TestRolloutPDBRestart (46.47s) PASS test/e2e.TestFunctionalSuite/TestRolloutPauseDurationGreaterThanProgressDeadlineSeconds (36.17s) PASS test/e2e.TestFunctionalSuite/TestRolloutPodVolumesItemsSpec (9.09s) PASS test/e2e.TestFunctionalSuite/TestRolloutRestart (18.80s) PASS test/e2e.TestFunctionalSuite/TestScaleDownOnSuccess (5.22s) PASS test/e2e.TestFunctionalSuite/TestScaleDownProgressively (5.02s) PASS test/e2e.TestFunctionalSuite/TestWorkloadRef (17.22s) PASS test/e2e.TestFunctionalSuite/TestWorkloadRefTemplate (2.79s) === RUN TestFunctionalSuite time="2024-12-04T06:17:49Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" --- FAIL: TestFunctionalSuite (403.65s) FAIL test/e2e.TestFunctionalSuite (403.65s) PASS test/e2e.TestHeaderRoutingSuite/TestIstioHostHeaderRoute (13.71s) PASS test/e2e.TestHeaderRoutingSuite (14.12s) PASS test/e2e.TestIstioSuite/TestIstioAbortUpdate (7.77s) PASS test/e2e.TestIstioSuite/TestIstioAbortUpdateDeleteAllCanaryPods (8.13s) PASS test/e2e.TestIstioSuite/TestIstioHostSplit (17.20s) PASS test/e2e.TestIstioSuite/TestIstioHostSplitExperimentStep (19.04s) PASS test/e2e.TestIstioSuite/TestIstioHostSplitOnlyTls (9.12s) PASS test/e2e.TestIstioSuite/TestIstioPingPongUpdate (12.48s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplit (11.16s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplitExperimentStep (19.31s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplitInStableDownscaleAfterCanaryAbort (8.05s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplitSingleRoute (8.71s) PASS test/e2e.TestIstioSuite/TestIstioUpdateInMiddleZeroCanaryReplicas (8.89s) PASS test/e2e.TestIstioSuite (130.31s) PASS test/e2e.TestMirrorRouteSuite/TestIstioHostMirrorRoute (13.68s) PASS test/e2e.TestMirrorRouteSuite (14.09s) PASS test/e2e.TestRollbackSuite/TestRollbackAnalysisOutsideWindow (32.13s) PASS test/e2e.TestRollbackSuite/TestRollbackAnalysisWithinWindow (23.08s) PASS test/e2e.TestRollbackSuite (57.92s) FAIL test/e2e DONE 111 tests, 6 skipped, 8 failures in 2571.530s === RUN TestAPISIXSuite/TestAPISIXCanarySetHeaderStep time="2024-12-04T06:28:24Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:28:25Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:26Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:27Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route configured\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:29Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:29Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:28:29Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:28:29Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.37s) FAIL test/e2e.TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 1) (5.37s) === RUN TestAPISIXSuite --- FAIL: TestAPISIXSuite (5.83s) FAIL test/e2e.TestAPISIXSuite (re-run 1) (5.83s) FAIL test/e2e === RUN TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress time="2024-12-04T06:28:41Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" time="2024-12-04T06:28:42Z" level=info msg="service/alb-rollout-root created\n" rollout=alb-rollout time="2024-12-04T06:28:43Z" level=info msg="service/alb-rollout-canary created\n" rollout=alb-rollout time="2024-12-04T06:28:44Z" level=info msg="service/alb-rollout-stable created\n" rollout=alb-rollout time="2024-12-04T06:28:46Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-1 created\n" rollout=alb-rollout time="2024-12-04T06:28:47Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-2 created\n" rollout=alb-rollout time="2024-12-04T06:28:48Z" level=info msg="rollout.argoproj.io/alb-rollout created\n" rollout=alb-rollout time="2024-12-04T06:28:48Z" level=info msg="Waiting for condition: status=Healthy" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Condition 'status=Healthy' met after 1s" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Experiment expectation 'experiment count == 0' met" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Updated rollout pod spec: 2024-12-04T06:28:49.515010095Z" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Sleeping 10s" rollout=alb-rollout require.go:72: Error Trace: /root/go/pkg/mod/github.com/tj/assert@v0.0.3/require.go:72 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:365 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/fixtures/then.go:29 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:354 Error: Not equal: expected: "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" actual : "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" Diff: --- Expected +++ Actual @@ -1 +1 @@ -{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} +{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServicePort":"80","Weight":20},{"ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} Test: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress Name: alb-rollout Namespace: argo-rollouts Status: ◌ Progressing Message: more replicas need to be updated Strategy: Canary Step: 0/1 SetWeight: 0 ActualWeight: 0 Images: nginx:1.19-alpine (stable, Σ:experiment-alb-canary, Σ:experiment-alb-stable) Replicas: Desired: 1 Current: 1 Updated: 0 Ready: 1 Available: 1 NAME KIND STATUS AGE INFO ⟳ alb-rollout Rollout ◌ Progressing 11s ├──# revision:2 │ ├──⧉ alb-rollout-955857848 ReplicaSet • ScaledDown 10s canary │ └──Σ alb-rollout-955857848-2-0 Experiment ◌ Running 10s │ ├──⧉ alb-rollout-955857848-2-0-experiment-alb-canary ReplicaSet ✔ Healthy 10s │ │ └──□ alb-rollout-955857848-2-0-experiment-alb-canary-zzkdh Pod ✔ Running 10s ready:1/1 │ └──⧉ alb-rollout-955857848-2-0-experiment-alb-stable ReplicaSet ✔ Healthy 10s │ └──□ alb-rollout-955857848-2-0-experiment-alb-stable-8qj8m Pod ✔ Running 10s ready:1/1 └──# revision:1 └──⧉ alb-rollout-d6f4fdcdc ReplicaSet ✔ Healthy 11s stable └──□ alb-rollout-d6f4fdcdc-4pvtc Pod ✔ Running 11s ready:1/1 --- apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: annotations: rollout.argoproj.io/revision: "2" creationTimestamp: "2024-12-04T06:28:48Z" generation: 3 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestALBExperimentStepNoSetWeightMultiIngress name: alb-rollout namespace: argo-rollouts resourceVersion: "2998685" uid: 5572890d-f1c1-4777-a672-700ff12b5c9a spec: replicas: 1 selector: matchLabels: app: alb-rollout strategy: canary: canaryService: alb-rollout-canary stableService: alb-rollout-stable steps: - experiment: analysisRunMetadata: {} duration: 15s templates: - metadata: {} name: experiment-alb-canary specRef: canary weight: 20 - metadata: {} name: experiment-alb-stable specRef: stable weight: 20 trafficRouting: alb: ingresses: - alb-rollout-multi-ingress-1 - alb-rollout-multi-ingress-2 rootService: alb-rollout-root servicePort: 80 template: metadata: annotations: update: "2024-12-04T06:28:49.515010095Z" creationTimestamp: null labels: app: alb-rollout spec: containers: - image: nginx:1.19-alpine name: alb-rollout ports: - containerPort: 80 name: http protocol: TCP resources: requests: cpu: 5m memory: 16Mi status: HPAReplicas: 1 availableReplicas: 1 blueGreen: {} canary: currentExperiment: alb-rollout-955857848-2-0 weights: additional: - weight: 20 - weight: 20 canary: podTemplateHash: "955857848" serviceName: alb-rollout-canary weight: 0 stable: podTemplateHash: d6f4fdcdc serviceName: alb-rollout-stable weight: 60 conditions: - lastTransitionTime: "2024-12-04T06:28:49Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: Rollout has minimum availability reason: AvailableReason status: "True" type: Available - lastTransitionTime: "2024-12-04T06:28:49Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: Rollout is not healthy reason: RolloutHealthy status: "False" type: Healthy - lastTransitionTime: "2024-12-04T06:28:49Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: RolloutCompleted reason: RolloutCompleted status: "False" type: Completed - lastTransitionTime: "2024-12-04T06:28:48Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: Created new replica set "alb-rollout-955857848" reason: NewReplicaSetCreated status: "True" type: Progressing currentPodHash: "955857848" currentStepHash: 8b59978 currentStepIndex: 0 message: more replicas need to be updated observedGeneration: "3" phase: Progressing readyReplicas: 1 replicas: 1 selector: app=alb-rollout stableRS: d6f4fdcdc 2024-12-04T06:28:48Z Normal RolloutAddedToInformer Rollout resource added to informer: argo-rollouts/alb-rollout 2024-12-04T06:28:48Z Normal RolloutNotCompleted Rollout not completed, started update to revision 1 (d6f4fdcdc) 2024-12-04T06:28:48Z Normal RolloutUpdated Rollout updated to revision 1 2024-12-04T06:28:48Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-d6f4fdcdc (revision 1) 2024-12-04T06:28:50Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-1` to desiredWeight '0' 2024-12-04T06:28:50Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-2` to desiredWeight '0' 2024-12-04T06:28:48Z Normal TrafficWeightUpdated Traffic weight updated to 0 2024-12-04T06:28:48Z Normal ScalingReplicaSet Scaled up ReplicaSet alb-rollout-d6f4fdcdc (revision 1) from 0 to 1 2024-12-04T06:28:48Z Normal RolloutCompleted Rollout completed update to revision 1 (d6f4fdcdc): Initial deploy 2024-12-04T06:28:49Z (x2) Normal TrafficWeightUpdated Traffic weight updated 2024-12-04T06:28:49Z Normal SwitchService Switched selector for service 'alb-rollout-stable' from '' to 'd6f4fdcdc' 2024-12-04T06:28:49Z Normal SwitchService Switched selector for service 'alb-rollout-canary' from '' to 'd6f4fdcdc' 2024-12-04T06:28:49Z Normal RolloutNotCompleted Rollout not completed, started update to revision 2 (955857848) 2024-12-04T06:28:49Z Normal RolloutUpdated Rollout updated to revision 2 2024-12-04T06:28:49Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-955857848 (revision 2) 2024-12-04T06:28:49Z Normal ExperimentCreated Created Experiment 'alb-rollout-955857848-2-0' 2024-12-04T06:28:50Z Normal TrafficWeightUpdated Traffic weight updated additional: [{20 } {20 }] time="2024-12-04T06:28:59Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" time="2024-12-04T06:29:00Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (19.10s) FAIL test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (re-run 1) (19.10s) === RUN TestAWSSuite --- FAIL: TestAWSSuite (19.55s) FAIL test/e2e.TestAWSSuite (re-run 1) (19.55s) FAIL test/e2e === RUN TestExperimentSuite/TestExperimentWithDryRunMetrics time="2024-12-04T06:29:11Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:29:13Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:29:14Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:29:14Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:29:45Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:29:45Z" level=info msg="Sleeping 3s" time="2024-12-04T06:29:48Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:29:48Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:29:48Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-tvztc Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 42b48566-723e-4428-8397-233bc437531f.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:29:14Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "2999522" uid: 0a9fa6ca-7ce2-434a-90b5-2a2d5426ecaa spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:29:15Z" conditions: - lastTransitionTime: "2024-12-04T06:29:45Z" lastUpdateTime: "2024-12-04T06:29:45Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:29:45Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:29:14Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:29:14Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:29:14Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:29:15Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:29:15Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:29:15Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:29:15Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:29:45Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:29:45Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:29:45Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:29:48Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:29:48Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.70s) FAIL test/e2e.TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 1) (36.70s) === RUN TestExperimentSuite --- FAIL: TestExperimentSuite (37.15s) FAIL test/e2e.TestExperimentSuite (re-run 1) (37.15s) FAIL test/e2e === RUN TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:30:01Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:30:01Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 159 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc0005829c0, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc0000c6300) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000573d00?, 0xc0008fcb18?, 0xc000aec3c0?}, {0x3c1144c, 0x4}, {0xc0009f5e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000573d00?, 0xc0008fcb18?, 0xc0000c6300?}, {0xc0009f5e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc0005829c0) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc0005829c0, 0xc0009ff170) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 150 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:30:02Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.75s) FAIL test/e2e.TestFunctionalSuite/TestControllerMetrics (re-run 1) (0.75s) === RUN TestFunctionalSuite time="2024-12-04T06:30:01Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" --- FAIL: TestFunctionalSuite (2.43s) FAIL test/e2e.TestFunctionalSuite (re-run 1) (2.43s) FAIL test/e2e DONE 2 runs, 119 tests, 6 skipped, 16 failures in 2685.020s === RUN TestAPISIXSuite/TestAPISIXCanarySetHeaderStep time="2024-12-04T06:30:13Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:30:15Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:16Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:17Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:18Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:18Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:30:18Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:30:18Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.27s) FAIL test/e2e.TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 2) (5.27s) === RUN TestAPISIXSuite --- FAIL: TestAPISIXSuite (5.73s) FAIL test/e2e.TestAPISIXSuite (re-run 2) (5.73s) FAIL test/e2e PASS test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (re-run 2) (21.81s) PASS test/e2e.TestAWSSuite (re-run 2) (22.29s) PASS test/e2e === RUN TestExperimentSuite/TestExperimentWithDryRunMetrics time="2024-12-04T06:31:04Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:31:05Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:31:06Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:31:06Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:31:37Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:31:37Z" level=info msg="Sleeping 3s" time="2024-12-04T06:31:40Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:31:40Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:31:40Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-qkbkf Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ ce9bf67c-d904-4dd0-857f-347bb81fd6ed.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:31:06Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3001376" uid: 77858e4f-d8e7-4c5c-8df3-248029763530 spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:31:07Z" conditions: - lastTransitionTime: "2024-12-04T06:31:37Z" lastUpdateTime: "2024-12-04T06:31:37Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:31:37Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:31:06Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:31:06Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:31:06Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:31:07Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:31:07Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:31:07Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:31:07Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:31:37Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:31:37Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:31:37Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:31:40Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:31:40Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.64s) FAIL test/e2e.TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 2) (36.64s) === RUN TestExperimentSuite --- FAIL: TestExperimentSuite (37.17s) FAIL test/e2e.TestExperimentSuite (re-run 2) (37.17s) FAIL test/e2e === RUN TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:31:53Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:31:53Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 155 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc000a02680, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc000c90c00) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc0000d8b00?, 0xc000c842e8?, 0xc000afc7d0?}, {0x3c1144c, 0x4}, {0xc000bf7e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc0000d8b00?, 0xc000c842e8?, 0xc000c90c00?}, {0xc000bf7e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc000a02680) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc000a02680, 0xc000c64ea0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 126 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:31:54Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.76s) FAIL test/e2e.TestFunctionalSuite/TestControllerMetrics (re-run 2) (0.76s) === RUN TestFunctionalSuite time="2024-12-04T06:31:53Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" --- FAIL: TestFunctionalSuite (2.47s) FAIL test/e2e.TestFunctionalSuite (re-run 2) (2.47s) FAIL test/e2e DONE 3 runs, 127 tests, 6 skipped, 22 failures in 2797.242s === RUN TestAPISIXSuite/TestAPISIXCanarySetHeaderStep time="2024-12-04T06:32:05Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:32:07Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:08Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:09Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:10Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:10Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:32:10Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:32:10Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.24s) FAIL test/e2e.TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 3) (5.24s) === RUN TestAPISIXSuite --- FAIL: TestAPISIXSuite (5.69s) FAIL test/e2e.TestAPISIXSuite (re-run 3) (5.69s) FAIL test/e2e === RUN TestExperimentSuite/TestExperimentWithDryRunMetrics time="2024-12-04T06:32:22Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:32:24Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:32:25Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:32:25Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:32:55Z" level=info msg="Condition 'phase=Successful' met after 29s" time="2024-12-04T06:32:55Z" level=info msg="Sleeping 3s" time="2024-12-04T06:32:58Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:32:58Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:32:58Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 33s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 33s delay:26s │ └──□ experiment-with-dry-run-baseline-bpfm9 Pod ✔ Running 33s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 8cf89b75-320b-449d-b8c4-9f65659755ed.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:32:25Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3002443" uid: 8acad0e7-5159-4a84-b6ca-7b3edf3ad29a spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:32:25Z" conditions: - lastTransitionTime: "2024-12-04T06:32:55Z" lastUpdateTime: "2024-12-04T06:32:55Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:32:55Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:32:25Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:32:25Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:32:25Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:32:25Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:32:25Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:32:25Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:32:25Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:32:55Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:32:55Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:32:55Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:32:58Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:32:58Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.02s) FAIL test/e2e.TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 3) (36.02s) === RUN TestExperimentSuite --- FAIL: TestExperimentSuite (36.46s) FAIL test/e2e.TestExperimentSuite (re-run 3) (36.46s) FAIL test/e2e === RUN TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:33:11Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:33:11Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 118 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc000e68000, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc000758700) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000669140?, 0xc000c0c240?, 0xc000e050d0?}, {0x3c1144c, 0x4}, {0xc000c9be70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000669140?, 0xc000c0c240?, 0xc000758700?}, {0xc000c9be70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc000e68000) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc000e68000, 0xc000e165a0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 144 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:33:12Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.74s) FAIL test/e2e.TestFunctionalSuite/TestControllerMetrics (re-run 3) (0.74s) === RUN TestFunctionalSuite time="2024-12-04T06:33:11Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" --- FAIL: TestFunctionalSuite (2.44s) FAIL test/e2e.TestFunctionalSuite (re-run 3) (2.44s) FAIL test/e2e DONE 4 runs, 133 tests, 6 skipped, 28 failures in 2875.239s === RUN TestAPISIXSuite/TestAPISIXCanarySetHeaderStep time="2024-12-04T06:33:23Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:33:25Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:26Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:27Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:28Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:28Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:33:28Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:33:29Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.27s) FAIL test/e2e.TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 4) (5.27s) === RUN TestAPISIXSuite --- FAIL: TestAPISIXSuite (5.73s) FAIL test/e2e.TestAPISIXSuite (re-run 4) (5.73s) FAIL test/e2e === RUN TestExperimentSuite/TestExperimentWithDryRunMetrics time="2024-12-04T06:33:40Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:33:42Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:33:43Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:33:43Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:34:15Z" level=info msg="Condition 'phase=Successful' met after 31s" time="2024-12-04T06:34:15Z" level=info msg="Sleeping 3s" time="2024-12-04T06:34:18Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:34:18Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:34:18Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 35s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 35s delay:26s │ └──□ experiment-with-dry-run-baseline-bbhg5 Pod ✔ Running 35s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 2c75d25d-66e9-4ac1-b1f7-bd9d1820f10f.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:33:43Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3003584" uid: 70952eed-51bb-4e58-931e-4716d8adbe18 spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:33:45Z" conditions: - lastTransitionTime: "2024-12-04T06:34:15Z" lastUpdateTime: "2024-12-04T06:34:15Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:34:15Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:33:43Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:33:43Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:33:43Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:33:45Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:33:45Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:33:45Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:33:45Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:34:15Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:34:15Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:34:15Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:34:18Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:34:18Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (37.67s) FAIL test/e2e.TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 4) (37.67s) === RUN TestExperimentSuite --- FAIL: TestExperimentSuite (38.13s) FAIL test/e2e.TestExperimentSuite (re-run 4) (38.13s) FAIL test/e2e === RUN TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:34:31Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:34:31Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 159 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc000a029c0, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc0008abb00) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000b92840?, 0xc000b8a560?, 0xc00059ed10?}, {0x3c1144c, 0x4}, {0xc00099fe70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000b92840?, 0xc000b8a560?, 0xc0008abb00?}, {0xc00099fe70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc000a029c0) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc000a029c0, 0xc000da3cb0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 177 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:34:32Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.73s) FAIL test/e2e.TestFunctionalSuite/TestControllerMetrics (re-run 4) (0.73s) === RUN TestFunctionalSuite time="2024-12-04T06:34:31Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" --- FAIL: TestFunctionalSuite (2.37s) FAIL test/e2e.TestFunctionalSuite (re-run 4) (2.37s) FAIL test/e2e DONE 5 runs, 139 tests, 6 skipped, 34 failures in 2954.963s === RUN TestAPISIXSuite/TestAPISIXCanarySetHeaderStep time="2024-12-04T06:34:43Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:34:45Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:46Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:47Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:48Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:48Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:34:48Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:34:49Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.58s) FAIL test/e2e.TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 5) (5.58s) === RUN TestAPISIXSuite --- FAIL: TestAPISIXSuite (6.05s) FAIL test/e2e.TestAPISIXSuite (re-run 5) (6.05s) FAIL test/e2e === RUN TestExperimentSuite/TestExperimentWithDryRunMetrics time="2024-12-04T06:35:01Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:35:02Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:35:03Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:35:03Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:35:34Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:35:34Z" level=info msg="Sleeping 3s" time="2024-12-04T06:35:37Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:35:37Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:35:37Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-ck5tp Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 3d7e136b-58df-4210-8ab9-c5155f4b5936.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:35:03Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3004686" uid: f0ad5a0d-8960-47a6-9b1a-619d09dd4e33 spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:35:04Z" conditions: - lastTransitionTime: "2024-12-04T06:35:34Z" lastUpdateTime: "2024-12-04T06:35:34Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:35:34Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:35:03Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:35:03Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:35:03Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:35:04Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:35:04Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:35:04Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:35:04Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:35:34Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:35:34Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:35:34Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:35:37Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:35:37Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.61s) FAIL test/e2e.TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 5) (36.61s) === RUN TestExperimentSuite --- FAIL: TestExperimentSuite (37.11s) FAIL test/e2e.TestExperimentSuite (re-run 5) (37.11s) FAIL test/e2e === RUN TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:35:50Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:35:50Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 126 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc0009896c0, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc0008b6700) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000b7f000?, 0xc000a5ef90?, 0xc000c2dfd0?}, {0x3c1144c, 0x4}, {0xc000c93e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000b7f000?, 0xc000a5ef90?, 0xc0008b6700?}, {0xc000c93e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc0009896c0) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc0009896c0, 0xc000a7ab40) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 134 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:35:51Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.75s) FAIL test/e2e.TestFunctionalSuite/TestControllerMetrics (re-run 5) (0.75s) === RUN TestFunctionalSuite time="2024-12-04T06:35:50Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" --- FAIL: TestFunctionalSuite (2.58s) FAIL test/e2e.TestFunctionalSuite (re-run 5) (2.58s) FAIL test/e2e === Skipped === SKIP: test/e2e TestAppMeshSuite (0.06s) === SKIP: test/e2e TestAWSSuite/TestALBBlueGreenUpdate (0.70s) time="2024-12-04T06:03:12Z" level=info msg="Deleting e2e-test-name=TestALBBlueGreenUpdate" time="2024-12-04T06:03:12Z" level=info msg="Deleting e2e-test-name=TestALBBlueGreenUpdate" --- SKIP: TestAWSSuite/TestALBBlueGreenUpdate (0.70s) === SKIP: test/e2e TestAWSSuite/TestALBCanaryUpdate (0.70s) time="2024-12-04T06:03:12Z" level=info msg="Deleting e2e-test-name=TestALBCanaryUpdate" time="2024-12-04T06:03:13Z" level=info msg="Deleting e2e-test-name=TestALBCanaryUpdate" --- SKIP: TestAWSSuite/TestALBCanaryUpdate (0.70s) === SKIP: test/e2e TestAWSSuite/TestALBCanaryUpdateMultiIngress (0.70s) time="2024-12-04T06:03:13Z" level=info msg="Deleting e2e-test-name=TestALBCanaryUpdateMultiIngress" time="2024-12-04T06:03:13Z" level=info msg="Deleting e2e-test-name=TestALBCanaryUpdateMultiIngress" --- SKIP: TestAWSSuite/TestALBCanaryUpdateMultiIngress (0.70s) === SKIP: test/e2e TestSMIIngressSuite (0.06s) === SKIP: test/e2e TestSMISuite (0.06s) === Failed === FAIL: test/e2e TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.36s) time="2024-12-04T06:02:40Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:02:41Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:42Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:43Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:45Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:02:45Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:02:45Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.36s) === FAIL: test/e2e TestAPISIXSuite (31.90s) === FAIL: test/e2e TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (22.12s) time="2024-12-04T06:04:27Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" time="2024-12-04T06:04:29Z" level=info msg="service/alb-rollout-root created\n" rollout=alb-rollout time="2024-12-04T06:04:30Z" level=info msg="service/alb-rollout-canary created\n" rollout=alb-rollout time="2024-12-04T06:04:32Z" level=info msg="service/alb-rollout-stable created\n" rollout=alb-rollout time="2024-12-04T06:04:34Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-1 created\n" rollout=alb-rollout time="2024-12-04T06:04:35Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-2 created\n" rollout=alb-rollout time="2024-12-04T06:04:37Z" level=info msg="rollout.argoproj.io/alb-rollout created\n" rollout=alb-rollout time="2024-12-04T06:04:37Z" level=info msg="Waiting for condition: status=Healthy" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Condition 'status=Healthy' met after 1s" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Experiment expectation 'experiment count == 0' met" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Updated rollout pod spec: 2024-12-04T06:04:38.668123905Z" rollout=alb-rollout time="2024-12-04T06:04:38Z" level=info msg="Sleeping 10s" rollout=alb-rollout require.go:72: Error Trace: /root/go/pkg/mod/github.com/tj/assert@v0.0.3/require.go:72 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:365 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/fixtures/then.go:29 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:354 Error: Not equal: expected: "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" actual : "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" Diff: --- Expected +++ Actual @@ -1 +1 @@ -{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} +{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServicePort":"80","Weight":20},{"ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} Test: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress Name: alb-rollout Namespace: argo-rollouts Status: ◌ Progressing Message: more replicas need to be updated Strategy: Canary Step: 0/1 SetWeight: 0 ActualWeight: 0 Images: nginx:1.19-alpine (stable, Σ:experiment-alb-canary, Σ:experiment-alb-stable) Replicas: Desired: 1 Current: 1 Updated: 0 Ready: 1 Available: 1 NAME KIND STATUS AGE INFO ⟳ alb-rollout Rollout ◌ Progressing 11s ├──# revision:2 │ ├──⧉ alb-rollout-84fd759574 ReplicaSet • ScaledDown 10s canary │ └──Σ alb-rollout-84fd759574-2-0 Experiment ◌ Running 10s │ ├──⧉ alb-rollout-84fd759574-2-0-experiment-alb-canary ReplicaSet ✔ Healthy 10s │ │ └──□ alb-rollout-84fd759574-2-0-experiment-alb-canary-kgj7q Pod ✔ Running 10s ready:1/1 │ └──⧉ alb-rollout-84fd759574-2-0-experiment-alb-stable ReplicaSet ✔ Healthy 10s │ └──□ alb-rollout-84fd759574-2-0-experiment-alb-stable-qgcrs Pod ✔ Running 10s ready:1/1 └──# revision:1 └──⧉ alb-rollout-d6f4fdcdc ReplicaSet ✔ Healthy 11s stable └──□ alb-rollout-d6f4fdcdc-v57jm Pod ✔ Running 11s ready:1/1 --- apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: annotations: rollout.argoproj.io/revision: "2" creationTimestamp: "2024-12-04T06:04:37Z" generation: 3 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestALBExperimentStepNoSetWeightMultiIngress name: alb-rollout namespace: argo-rollouts resourceVersion: "2965663" uid: 4e5112df-b5a8-481b-b707-222bedc53d92 spec: replicas: 1 selector: matchLabels: app: alb-rollout strategy: canary: canaryService: alb-rollout-canary stableService: alb-rollout-stable steps: - experiment: analysisRunMetadata: {} duration: 15s templates: - metadata: {} name: experiment-alb-canary specRef: canary weight: 20 - metadata: {} name: experiment-alb-stable specRef: stable weight: 20 trafficRouting: alb: ingresses: - alb-rollout-multi-ingress-1 - alb-rollout-multi-ingress-2 rootService: alb-rollout-root servicePort: 80 template: metadata: annotations: update: "2024-12-04T06:04:38.668123905Z" creationTimestamp: null labels: app: alb-rollout spec: containers: - image: nginx:1.19-alpine name: alb-rollout ports: - containerPort: 80 name: http protocol: TCP resources: requests: cpu: 5m memory: 16Mi status: HPAReplicas: 1 availableReplicas: 1 blueGreen: {} canary: currentExperiment: alb-rollout-84fd759574-2-0 weights: additional: - weight: 20 - weight: 20 canary: podTemplateHash: 84fd759574 serviceName: alb-rollout-canary weight: 0 stable: podTemplateHash: d6f4fdcdc serviceName: alb-rollout-stable weight: 60 conditions: - lastTransitionTime: "2024-12-04T06:04:38Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: Rollout has minimum availability reason: AvailableReason status: "True" type: Available - lastTransitionTime: "2024-12-04T06:04:38Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: Rollout is not healthy reason: RolloutHealthy status: "False" type: Healthy - lastTransitionTime: "2024-12-04T06:04:38Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: RolloutCompleted reason: RolloutCompleted status: "False" type: Completed - lastTransitionTime: "2024-12-04T06:04:37Z" lastUpdateTime: "2024-12-04T06:04:38Z" message: Created new replica set "alb-rollout-84fd759574" reason: NewReplicaSetCreated status: "True" type: Progressing currentPodHash: 84fd759574 currentStepHash: 8b59978 currentStepIndex: 0 message: more replicas need to be updated observedGeneration: "3" phase: Progressing readyReplicas: 1 replicas: 1 selector: app=alb-rollout stableRS: d6f4fdcdc 2024-12-04T06:04:37Z Normal RolloutAddedToInformer Rollout resource added to informer: argo-rollouts/alb-rollout 2024-12-04T06:04:37Z Normal RolloutNotCompleted Rollout not completed, started update to revision 1 (d6f4fdcdc) 2024-12-04T06:04:37Z Normal RolloutUpdated Rollout updated to revision 1 2024-12-04T06:04:37Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-d6f4fdcdc (revision 1) 2024-12-04T06:04:39Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-1` to desiredWeight '0' 2024-12-04T06:04:39Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-2` to desiredWeight '0' 2024-12-04T06:04:37Z Normal TrafficWeightUpdated Traffic weight updated to 0 2024-12-04T06:04:37Z Normal ScalingReplicaSet Scaled up ReplicaSet alb-rollout-d6f4fdcdc (revision 1) from 0 to 1 2024-12-04T06:04:37Z Normal RolloutCompleted Rollout completed update to revision 1 (d6f4fdcdc): Initial deploy 2024-12-04T06:04:38Z (x2) Normal TrafficWeightUpdated Traffic weight updated 2024-12-04T06:04:38Z Normal SwitchService Switched selector for service 'alb-rollout-stable' from '' to 'd6f4fdcdc' 2024-12-04T06:04:38Z Normal SwitchService Switched selector for service 'alb-rollout-canary' from '' to 'd6f4fdcdc' 2024-12-04T06:04:38Z Normal RolloutNotCompleted Rollout not completed, started update to revision 2 (84fd759574) 2024-12-04T06:04:38Z Normal RolloutUpdated Rollout updated to revision 2 2024-12-04T06:04:38Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-84fd759574 (revision 2) 2024-12-04T06:04:38Z Normal ExperimentCreated Created Experiment 'alb-rollout-84fd759574-2-0' 2024-12-04T06:04:39Z Normal TrafficWeightUpdated Traffic weight updated additional: [{20 } {20 }] time="2024-12-04T06:04:48Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" --- FAIL: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (22.12s) === FAIL: test/e2e TestAWSSuite (185.42s) === FAIL: test/e2e TestExperimentSuite/TestExperimentWithDryRunMetrics (37.66s) time="2024-12-04T06:15:28Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:15:30Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:15:32Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:15:32Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:16:03Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:16:03Z" level=info msg="Sleeping 3s" time="2024-12-04T06:16:06Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:16:06Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:16:06Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-c4hmv Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 96cdda97-79ea-4423-ac31-3b683c4a4577.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:15:32Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "2980655" uid: 3a90e476-6753-4fdf-ad21-e6e6f89e958d spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:15:33Z" conditions: - lastTransitionTime: "2024-12-04T06:16:03Z" lastUpdateTime: "2024-12-04T06:16:03Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:16:03Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:15:32Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:15:32Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:15:32Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:15:33Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:15:33Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:15:33Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:15:33Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:16:03Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:16:03Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:16:03Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:16:06Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (37.66s) === FAIL: test/e2e TestExperimentSuite (129.31s) === FAIL: test/e2e TestFunctionalSuite/TestControllerMetrics (0.73s) time="2024-12-04T06:19:23Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:19:23Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 6322 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc002458340, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc000bd1200) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000660d00?, 0xc0010165a0?, 0xc001239690?}, {0x3c1144c, 0x4}, {0xc0019b3e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000660d00?, 0xc0010165a0?, 0xc000bd1200?}, {0xc0019b3e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc002458340) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc002458340, 0xc000c8cea0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 5379 /usr/local/go/src/testing/testing.go:1648 +0x3ad --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.73s) === FAIL: test/e2e TestFunctionalSuite (403.65s) time="2024-12-04T06:17:49Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" === FAIL: test/e2e TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 1) (5.37s) time="2024-12-04T06:28:24Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:28:25Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:26Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:27Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route configured\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:29Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:28:29Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:28:29Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:28:29Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.37s) === FAIL: test/e2e TestAPISIXSuite (re-run 1) (5.83s) === FAIL: test/e2e TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (re-run 1) (19.10s) time="2024-12-04T06:28:41Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" time="2024-12-04T06:28:42Z" level=info msg="service/alb-rollout-root created\n" rollout=alb-rollout time="2024-12-04T06:28:43Z" level=info msg="service/alb-rollout-canary created\n" rollout=alb-rollout time="2024-12-04T06:28:44Z" level=info msg="service/alb-rollout-stable created\n" rollout=alb-rollout time="2024-12-04T06:28:46Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-1 created\n" rollout=alb-rollout time="2024-12-04T06:28:47Z" level=info msg="Warning: annotation \"kubernetes.io/ingress.class\" is deprecated, please use 'spec.ingressClassName' instead\ningress.networking.k8s.io/alb-rollout-multi-ingress-2 created\n" rollout=alb-rollout time="2024-12-04T06:28:48Z" level=info msg="rollout.argoproj.io/alb-rollout created\n" rollout=alb-rollout time="2024-12-04T06:28:48Z" level=info msg="Waiting for condition: status=Healthy" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Condition 'status=Healthy' met after 1s" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Experiment expectation 'experiment count == 0' met" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Updated rollout pod spec: 2024-12-04T06:28:49.515010095Z" rollout=alb-rollout time="2024-12-04T06:28:49Z" level=info msg="Sleeping 10s" rollout=alb-rollout require.go:72: Error Trace: /root/go/pkg/mod/github.com/tj/assert@v0.0.3/require.go:72 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:365 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/fixtures/then.go:29 /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/aws_test.go:354 Error: Not equal: expected: "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" actual : "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}" Diff: --- Expected +++ Actual @@ -1 +1 @@ -{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} +{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServicePort":"80","Weight":20},{"ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}} Test: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress Name: alb-rollout Namespace: argo-rollouts Status: ◌ Progressing Message: more replicas need to be updated Strategy: Canary Step: 0/1 SetWeight: 0 ActualWeight: 0 Images: nginx:1.19-alpine (stable, Σ:experiment-alb-canary, Σ:experiment-alb-stable) Replicas: Desired: 1 Current: 1 Updated: 0 Ready: 1 Available: 1 NAME KIND STATUS AGE INFO ⟳ alb-rollout Rollout ◌ Progressing 11s ├──# revision:2 │ ├──⧉ alb-rollout-955857848 ReplicaSet • ScaledDown 10s canary │ └──Σ alb-rollout-955857848-2-0 Experiment ◌ Running 10s │ ├──⧉ alb-rollout-955857848-2-0-experiment-alb-canary ReplicaSet ✔ Healthy 10s │ │ └──□ alb-rollout-955857848-2-0-experiment-alb-canary-zzkdh Pod ✔ Running 10s ready:1/1 │ └──⧉ alb-rollout-955857848-2-0-experiment-alb-stable ReplicaSet ✔ Healthy 10s │ └──□ alb-rollout-955857848-2-0-experiment-alb-stable-8qj8m Pod ✔ Running 10s ready:1/1 └──# revision:1 └──⧉ alb-rollout-d6f4fdcdc ReplicaSet ✔ Healthy 11s stable └──□ alb-rollout-d6f4fdcdc-4pvtc Pod ✔ Running 11s ready:1/1 --- apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: annotations: rollout.argoproj.io/revision: "2" creationTimestamp: "2024-12-04T06:28:48Z" generation: 3 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestALBExperimentStepNoSetWeightMultiIngress name: alb-rollout namespace: argo-rollouts resourceVersion: "2998685" uid: 5572890d-f1c1-4777-a672-700ff12b5c9a spec: replicas: 1 selector: matchLabels: app: alb-rollout strategy: canary: canaryService: alb-rollout-canary stableService: alb-rollout-stable steps: - experiment: analysisRunMetadata: {} duration: 15s templates: - metadata: {} name: experiment-alb-canary specRef: canary weight: 20 - metadata: {} name: experiment-alb-stable specRef: stable weight: 20 trafficRouting: alb: ingresses: - alb-rollout-multi-ingress-1 - alb-rollout-multi-ingress-2 rootService: alb-rollout-root servicePort: 80 template: metadata: annotations: update: "2024-12-04T06:28:49.515010095Z" creationTimestamp: null labels: app: alb-rollout spec: containers: - image: nginx:1.19-alpine name: alb-rollout ports: - containerPort: 80 name: http protocol: TCP resources: requests: cpu: 5m memory: 16Mi status: HPAReplicas: 1 availableReplicas: 1 blueGreen: {} canary: currentExperiment: alb-rollout-955857848-2-0 weights: additional: - weight: 20 - weight: 20 canary: podTemplateHash: "955857848" serviceName: alb-rollout-canary weight: 0 stable: podTemplateHash: d6f4fdcdc serviceName: alb-rollout-stable weight: 60 conditions: - lastTransitionTime: "2024-12-04T06:28:49Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: Rollout has minimum availability reason: AvailableReason status: "True" type: Available - lastTransitionTime: "2024-12-04T06:28:49Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: Rollout is not healthy reason: RolloutHealthy status: "False" type: Healthy - lastTransitionTime: "2024-12-04T06:28:49Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: RolloutCompleted reason: RolloutCompleted status: "False" type: Completed - lastTransitionTime: "2024-12-04T06:28:48Z" lastUpdateTime: "2024-12-04T06:28:49Z" message: Created new replica set "alb-rollout-955857848" reason: NewReplicaSetCreated status: "True" type: Progressing currentPodHash: "955857848" currentStepHash: 8b59978 currentStepIndex: 0 message: more replicas need to be updated observedGeneration: "3" phase: Progressing readyReplicas: 1 replicas: 1 selector: app=alb-rollout stableRS: d6f4fdcdc 2024-12-04T06:28:48Z Normal RolloutAddedToInformer Rollout resource added to informer: argo-rollouts/alb-rollout 2024-12-04T06:28:48Z Normal RolloutNotCompleted Rollout not completed, started update to revision 1 (d6f4fdcdc) 2024-12-04T06:28:48Z Normal RolloutUpdated Rollout updated to revision 1 2024-12-04T06:28:48Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-d6f4fdcdc (revision 1) 2024-12-04T06:28:50Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-1` to desiredWeight '0' 2024-12-04T06:28:50Z (x2) Normal PatchingALBIngress Updating Ingress `alb-rollout-multi-ingress-2` to desiredWeight '0' 2024-12-04T06:28:48Z Normal TrafficWeightUpdated Traffic weight updated to 0 2024-12-04T06:28:48Z Normal ScalingReplicaSet Scaled up ReplicaSet alb-rollout-d6f4fdcdc (revision 1) from 0 to 1 2024-12-04T06:28:48Z Normal RolloutCompleted Rollout completed update to revision 1 (d6f4fdcdc): Initial deploy 2024-12-04T06:28:49Z (x2) Normal TrafficWeightUpdated Traffic weight updated 2024-12-04T06:28:49Z Normal SwitchService Switched selector for service 'alb-rollout-stable' from '' to 'd6f4fdcdc' 2024-12-04T06:28:49Z Normal SwitchService Switched selector for service 'alb-rollout-canary' from '' to 'd6f4fdcdc' 2024-12-04T06:28:49Z Normal RolloutNotCompleted Rollout not completed, started update to revision 2 (955857848) 2024-12-04T06:28:49Z Normal RolloutUpdated Rollout updated to revision 2 2024-12-04T06:28:49Z Normal NewReplicaSetCreated Created ReplicaSet alb-rollout-955857848 (revision 2) 2024-12-04T06:28:49Z Normal ExperimentCreated Created Experiment 'alb-rollout-955857848-2-0' 2024-12-04T06:28:50Z Normal TrafficWeightUpdated Traffic weight updated additional: [{20 } {20 }] time="2024-12-04T06:28:59Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress" time="2024-12-04T06:29:00Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (19.10s) === FAIL: test/e2e TestAWSSuite (re-run 1) (19.55s) === FAIL: test/e2e TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 1) (36.70s) time="2024-12-04T06:29:11Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:29:13Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:29:14Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:29:14Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:29:45Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:29:45Z" level=info msg="Sleeping 3s" time="2024-12-04T06:29:48Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:29:48Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:29:48Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-tvztc Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 42b48566-723e-4428-8397-233bc437531f.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:29:14Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "2999522" uid: 0a9fa6ca-7ce2-434a-90b5-2a2d5426ecaa spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:29:15Z" conditions: - lastTransitionTime: "2024-12-04T06:29:45Z" lastUpdateTime: "2024-12-04T06:29:45Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:29:45Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:29:14Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:29:14Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:29:14Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:29:15Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:29:15Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:29:15Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:29:15Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:29:45Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:29:45Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:29:45Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:29:48Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:29:48Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.70s) === FAIL: test/e2e TestExperimentSuite (re-run 1) (37.15s) === FAIL: test/e2e TestFunctionalSuite/TestControllerMetrics (re-run 1) (0.75s) time="2024-12-04T06:30:01Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:30:01Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 159 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc0005829c0, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc0000c6300) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000573d00?, 0xc0008fcb18?, 0xc000aec3c0?}, {0x3c1144c, 0x4}, {0xc0009f5e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000573d00?, 0xc0008fcb18?, 0xc0000c6300?}, {0xc0009f5e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc0005829c0) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc0005829c0, 0xc0009ff170) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 150 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:30:02Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.75s) === FAIL: test/e2e TestFunctionalSuite (re-run 1) (2.43s) time="2024-12-04T06:30:01Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" === FAIL: test/e2e TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 2) (5.27s) time="2024-12-04T06:30:13Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:30:15Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:16Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:17Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:18Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:30:18Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:30:18Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:30:18Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.27s) === FAIL: test/e2e TestAPISIXSuite (re-run 2) (5.73s) === FAIL: test/e2e TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 2) (36.64s) time="2024-12-04T06:31:04Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:31:05Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:31:06Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:31:06Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:31:37Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:31:37Z" level=info msg="Sleeping 3s" time="2024-12-04T06:31:40Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:31:40Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:31:40Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-qkbkf Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ ce9bf67c-d904-4dd0-857f-347bb81fd6ed.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:31:06Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3001376" uid: 77858e4f-d8e7-4c5c-8df3-248029763530 spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:31:07Z" conditions: - lastTransitionTime: "2024-12-04T06:31:37Z" lastUpdateTime: "2024-12-04T06:31:37Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:31:37Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:31:06Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:31:06Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:31:06Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:31:07Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:31:07Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:31:07Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:31:07Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:31:37Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:31:37Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:31:37Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:31:40Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:31:40Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.64s) === FAIL: test/e2e TestExperimentSuite (re-run 2) (37.17s) === FAIL: test/e2e TestFunctionalSuite/TestControllerMetrics (re-run 2) (0.76s) time="2024-12-04T06:31:53Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:31:53Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 155 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc000a02680, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc000c90c00) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc0000d8b00?, 0xc000c842e8?, 0xc000afc7d0?}, {0x3c1144c, 0x4}, {0xc000bf7e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc0000d8b00?, 0xc000c842e8?, 0xc000c90c00?}, {0xc000bf7e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc000a02680) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc000a02680, 0xc000c64ea0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 126 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:31:54Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.76s) === FAIL: test/e2e TestFunctionalSuite (re-run 2) (2.47s) time="2024-12-04T06:31:53Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" === FAIL: test/e2e TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 3) (5.24s) time="2024-12-04T06:32:05Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:32:07Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:08Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:09Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:10Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:32:10Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:32:10Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:32:10Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.24s) === FAIL: test/e2e TestAPISIXSuite (re-run 3) (5.69s) === FAIL: test/e2e TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 3) (36.02s) time="2024-12-04T06:32:22Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:32:24Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:32:25Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:32:25Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:32:55Z" level=info msg="Condition 'phase=Successful' met after 29s" time="2024-12-04T06:32:55Z" level=info msg="Sleeping 3s" time="2024-12-04T06:32:58Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:32:58Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:32:58Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 33s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 33s delay:26s │ └──□ experiment-with-dry-run-baseline-bpfm9 Pod ✔ Running 33s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 8cf89b75-320b-449d-b8c4-9f65659755ed.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:32:25Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3002443" uid: 8acad0e7-5159-4a84-b6ca-7b3edf3ad29a spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:32:25Z" conditions: - lastTransitionTime: "2024-12-04T06:32:55Z" lastUpdateTime: "2024-12-04T06:32:55Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:32:55Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:32:25Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:32:25Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:32:25Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:32:25Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:32:25Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:32:25Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:32:25Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:32:55Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:32:55Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:32:55Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:32:58Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:32:58Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.02s) === FAIL: test/e2e TestExperimentSuite (re-run 3) (36.46s) === FAIL: test/e2e TestFunctionalSuite/TestControllerMetrics (re-run 3) (0.74s) time="2024-12-04T06:33:11Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:33:11Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 118 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc000e68000, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc000758700) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000669140?, 0xc000c0c240?, 0xc000e050d0?}, {0x3c1144c, 0x4}, {0xc000c9be70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000669140?, 0xc000c0c240?, 0xc000758700?}, {0xc000c9be70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc000e68000) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc000e68000, 0xc000e165a0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 144 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:33:12Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.74s) === FAIL: test/e2e TestFunctionalSuite (re-run 3) (2.44s) time="2024-12-04T06:33:11Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" === FAIL: test/e2e TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 4) (5.27s) time="2024-12-04T06:33:23Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:33:25Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:26Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:27Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:28Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:33:28Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:33:28Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:33:29Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.27s) === FAIL: test/e2e TestAPISIXSuite (re-run 4) (5.73s) === FAIL: test/e2e TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 4) (37.67s) time="2024-12-04T06:33:40Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:33:42Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:33:43Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:33:43Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:34:15Z" level=info msg="Condition 'phase=Successful' met after 31s" time="2024-12-04T06:34:15Z" level=info msg="Sleeping 3s" time="2024-12-04T06:34:18Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:34:18Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:34:18Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 35s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 35s delay:26s │ └──□ experiment-with-dry-run-baseline-bbhg5 Pod ✔ Running 35s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 2c75d25d-66e9-4ac1-b1f7-bd9d1820f10f.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:33:43Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3003584" uid: 70952eed-51bb-4e58-931e-4716d8adbe18 spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:33:45Z" conditions: - lastTransitionTime: "2024-12-04T06:34:15Z" lastUpdateTime: "2024-12-04T06:34:15Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:34:15Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:33:43Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:33:43Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:33:43Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:33:45Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:33:45Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:33:45Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:33:45Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:34:15Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:34:15Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:34:15Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:34:18Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:34:18Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (37.67s) === FAIL: test/e2e TestExperimentSuite (re-run 4) (38.13s) === FAIL: test/e2e TestFunctionalSuite/TestControllerMetrics (re-run 4) (0.73s) time="2024-12-04T06:34:31Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:34:31Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 159 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc000a029c0, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc0008abb00) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000b92840?, 0xc000b8a560?, 0xc00059ed10?}, {0x3c1144c, 0x4}, {0xc00099fe70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000b92840?, 0xc000b8a560?, 0xc0008abb00?}, {0xc00099fe70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc000a029c0) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc000a029c0, 0xc000da3cb0) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 177 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:34:32Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.73s) === FAIL: test/e2e TestFunctionalSuite (re-run 4) (2.37s) time="2024-12-04T06:34:31Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" === FAIL: test/e2e TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (re-run 5) (5.58s) time="2024-12-04T06:34:43Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:34:45Z" level=info msg="service/rollout-apisix-canary-canary created\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:46Z" level=info msg="service/rollout-apisix-canary-stable created\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:47Z" level=info msg="apisixroute.apisix.apache.org/rollouts-apisix-route unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:48Z" level=info msg="rollout.argoproj.io/rollout-apisix-canary unchanged\n" rollout=rollout-apisix-canary time="2024-12-04T06:34:48Z" level=info msg="Waiting for condition: status=Healthy" rollout=rollout-apisix-canary when.go:423: rollouts.argoproj.io "rollout-apisix-canary" not found time="2024-12-04T06:34:48Z" level=info msg="Deleting e2e-test-name=TestAPISIXCanarySetHeaderStep" time="2024-12-04T06:34:49Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.58s) === FAIL: test/e2e TestAPISIXSuite (re-run 5) (6.05s) === FAIL: test/e2e TestExperimentSuite/TestExperimentWithDryRunMetrics (re-run 5) (36.61s) time="2024-12-04T06:35:01Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:35:02Z" level=info msg="analysistemplate.argoproj.io/dry-run-job created\n" time="2024-12-04T06:35:03Z" level=info msg="experiment.argoproj.io/experiment-with-dry-run created\n" time="2024-12-04T06:35:03Z" level=info msg="Waiting for Experiment experiment-with-dry-run condition: phase=Successful" time="2024-12-04T06:35:34Z" level=info msg="Condition 'phase=Successful' met after 30s" time="2024-12-04T06:35:34Z" level=info msg="Sleeping 3s" time="2024-12-04T06:35:37Z" level=info msg="Expected Dry-Run Summary: Count=1, Successful=0, Failed=1, Error=0, Inconclusive=0" time="2024-12-04T06:35:37Z" level=info msg="Analysis Dry-Run Summary: Count=1, Successful=1, Failed=0, Error=0, Inconclusive=0" time="2024-12-04T06:35:37Z" level=error msg="Dry-Run Summary of AnalysisRun: 'experiment-with-dry-run-dry-run-job' doesn't match the expectations" Name: experiment-with-dry-run Namespace: argo-rollouts Status: ✔ Successful Images: nginx:1.19-alpine (Σ:baseline) NAME KIND STATUS AGE INFO Σ experiment-with-dry-run Experiment ✔ Successful 34s ├──⧉ experiment-with-dry-run-baseline ReplicaSet ✔ Healthy 34s delay:26s │ └──□ experiment-with-dry-run-baseline-ck5tp Pod ✔ Running 34s ready:1/1 └──α experiment-with-dry-run-dry-run-job AnalysisRun ✔ Successful 33s ✔ 1 └──⊞ 3d7e136b-58df-4210-8ab9-c5155f4b5936.test-1.1 Job ✔ Successful 33s --- apiVersion: argoproj.io/v1alpha1 kind: Experiment metadata: creationTimestamp: "2024-12-04T06:35:03Z" generation: 9 labels: argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e e2e-test-name: TestExperimentWithDryRunMetrics name: experiment-with-dry-run namespace: argo-rollouts resourceVersion: "3004686" uid: f0ad5a0d-8960-47a6-9b1a-619d09dd4e33 spec: analyses: - name: dry-run-job templateName: dry-run-job analysisRunMetadata: {} dryRun: - metricName: test.* duration: 30s progressDeadlineSeconds: 30 templates: - name: baseline replicas: 1 selector: matchLabels: app: experiment-with-dry-run service: {} template: metadata: creationTimestamp: null labels: app: experiment-with-dry-run spec: containers: - image: nginx:1.19-alpine name: experiment-with-dry-run resources: requests: cpu: 1m memory: 16Mi status: analysisRuns: - analysisRun: experiment-with-dry-run-dry-run-job name: dry-run-job phase: Running availableAt: "2024-12-04T06:35:04Z" conditions: - lastTransitionTime: "2024-12-04T06:35:34Z" lastUpdateTime: "2024-12-04T06:35:34Z" message: Experiment "experiment-with-dry-run" has successfully ran and completed. reason: ExperimentCompleted status: "False" type: Progressing phase: Successful templateStatuses: - availableReplicas: 1 lastTransitionTime: "2024-12-04T06:35:34Z" name: baseline readyReplicas: 1 replicas: 1 status: Successful updatedReplicas: 1 2024-12-04T06:35:03Z (x2) Normal TemplateProgressing Template 'baseline' transitioned from -> Progressing 2024-12-04T06:35:03Z (x2) Normal ExperimentPending Experiment transitioned from -> Pending 2024-12-04T06:35:03Z Normal ScalingReplicaSet Scaled up ReplicaSet experiment-with-dry-run-baseline from 0 to 1 2024-12-04T06:35:04Z Normal TemplateRunning Template 'baseline' transitioned from Progressing -> Running 2024-12-04T06:35:04Z Normal ExperimentRunning Experiment transitioned from Pending -> Running 2024-12-04T06:35:04Z Normal AnalysisRunPending AnalysisRun 'dry-run-job' transitioned from -> Pending 2024-12-04T06:35:04Z Normal AnalysisRunRunning AnalysisRun 'dry-run-job' transitioned from -> Running 2024-12-04T06:35:34Z Normal TemplateSuccessful Template 'baseline' transitioned from Running -> Successful 2024-12-04T06:35:34Z Normal ExperimentSuccessful Experiment transitioned from Running -> Successful 2024-12-04T06:35:34Z Normal AnalysisRunTerminating Terminating dry-run-job (experiment-with-dry-run-dry-run-job) time="2024-12-04T06:35:37Z" level=info msg="Deleting e2e-test-name=TestExperimentWithDryRunMetrics" time="2024-12-04T06:35:37Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.61s) === FAIL: test/e2e TestExperimentSuite (re-run 5) (37.11s) === FAIL: test/e2e TestFunctionalSuite/TestControllerMetrics (re-run 5) (0.75s) time="2024-12-04T06:35:50Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" functional_test.go:1407: Error Trace: /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1407 Error: Received unexpected error: Get "http://localhost:8090/metrics": dial tcp [::1]:8090: connect: connection refused Test: TestFunctionalSuite/TestControllerMetrics time="2024-12-04T06:35:50Z" level=info msg="Deleting e2e-test-name=TestControllerMetrics" panic.go:261: test panicked: runtime error: invalid memory address or nil pointer dereference goroutine 126 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x5e github.com/stretchr/testify/suite.failOnPanic(0xc0009896c0, {0x37dde20, 0x5c66ac0}) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:89 +0x37 github.com/stretchr/testify/suite.Run.func1.1() /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:188 +0x25d panic({0x37dde20?, 0x5c66ac0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/argoproj/argo-rollouts/test/e2e.(*FunctionalSuite).TestControllerMetrics(0xc0008b6700) /tmp/tmp.ghLtMCCrSg/argo-rollouts/test/e2e/functional_test.go:1408 +0xc4 reflect.Value.call({0xc000b7f000?, 0xc000a5ef90?, 0xc000c2dfd0?}, {0x3c1144c, 0x4}, {0xc000c93e70, 0x1, 0x3405488?}) /usr/local/go/src/reflect/value.go:596 +0xce7 reflect.Value.Call({0xc000b7f000?, 0xc000a5ef90?, 0xc0008b6700?}, {0xc000c93e70?, 0x154fc4d?, 0x5218ea8?}) /usr/local/go/src/reflect/value.go:380 +0xb9 github.com/stretchr/testify/suite.Run.func1(0xc0009896c0) /root/go/pkg/mod/github.com/stretchr/testify@v1.9.0/suite/suite.go:202 +0x467 testing.tRunner(0xc0009896c0, 0xc000a7ab40) /usr/local/go/src/testing/testing.go:1595 +0xff created by testing.(*T).Run in goroutine 134 /usr/local/go/src/testing/testing.go:1648 +0x3ad time="2024-12-04T06:35:51Z" level=info msg="Deleting e2e-test-name" --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.75s) === FAIL: test/e2e TestFunctionalSuite (re-run 5) (2.58s) time="2024-12-04T06:35:50Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n" DONE 6 runs, 145 tests, 6 skipped, 40 failures in 3034.276s make: *** [Makefile:239: test-e2e] Error 1 + set +x ----------------------------------------------------------------- These were the tests that succeeded: PASS test/e2e PASS test/e2e.TestAPISIXSuite/TestAPISIXCanaryStep (26.13s) PASS test/e2e.TestAWSSuite (re-run 2) (22.29s) PASS test/e2e.TestAWSSuite/TestALBExperimentStep (23.86s) PASS test/e2e.TestAWSSuite/TestALBExperimentStepMultiIngress (25.14s) PASS test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeight (24.14s) PASS test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (re-run 2) (21.81s) PASS test/e2e.TestAWSSuite/TestALBMesh_PingPong_StableCanary_Update (19.70s) PASS test/e2e.TestAWSSuite/TestALBPingPongUpdate (15.94s) PASS test/e2e.TestAWSSuite/TestALBPingPongUpdateMultiIngress (17.02s) PASS test/e2e.TestAWSSuite/TestAlbHeaderRoute (10.54s) PASS test/e2e.TestAWSSuite/TestAlbHeaderRouteMultiIngress (24.46s) PASS test/e2e.TestAnalysisSuite (306.93s) PASS test/e2e.TestAnalysisSuite/TestAnalysisWithArgs (39.95s) PASS test/e2e.TestAnalysisSuite/TestAnalysisWithSecret (29.81s) PASS test/e2e.TestAnalysisSuite/TestBackgroundAnalysisWithArgs (28.46s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenAbortAndUpdate (21.17s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenAnalysis (28.79s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenKitchenSink (44.89s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenPostPromotionFail (19.50s) PASS test/e2e.TestAnalysisSuite/TestBlueGreenPrePromotionFail (29.97s) PASS test/e2e.TestAnalysisSuite/TestCanaryBackgroundAnalysis (4.33s) PASS test/e2e.TestAnalysisSuite/TestCanaryFailInlineMultipleAnalysis (11.66s) PASS test/e2e.TestAnalysisSuite/TestCanaryInlineAnalysis (16.07s) PASS test/e2e.TestAnalysisSuite/TestCanaryInlineMultipleAnalysis (18.05s) PASS test/e2e.TestAnalysisSuite/TestMultipleAnalysis (6.19s) PASS test/e2e.TestBlueGreenSuite (146.82s) PASS test/e2e.TestBlueGreenSuite/TestBlueGreenPreviewReplicaCount (8.12s) PASS test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithPause (64.26s) PASS test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithoutPause (64.13s) PASS test/e2e.TestBlueGreenSuite/TestEphemeralMetadata (9.91s) PASS test/e2e.TestCanarySuite (394.08s) PASS test/e2e.TestCanarySuite/TestCanaryDynamicStableScale (25.96s) PASS test/e2e.TestCanarySuite/TestCanaryDynamicStableScaleRollbackToStable (18.01s) PASS test/e2e.TestCanarySuite/TestCanaryProgressDeadlineExceededWithPause (124.16s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownDelay (13.76s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownDelayWithProgressDeadline (18.86s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbort (12.17s) PASS test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbortNoTrafficRouting (11.03s) PASS test/e2e.TestCanarySuite/TestCanarySetCanaryScale (10.90s) PASS test/e2e.TestCanarySuite/TestCanaryUnScaleDownOnAbort (14.17s) PASS test/e2e.TestCanarySuite/TestCanaryWithPausedRollout (102.84s) PASS test/e2e.TestCanarySuite/TestEphemeralMetadata (7.75s) PASS test/e2e.TestCanarySuite/TestReduceWeightAndHonorMaxUnavailable (30.01s) PASS test/e2e.TestCanarySuite/TestRolloutScalingDuringUpdate (3.71s) PASS test/e2e.TestCanarySuite/TestRolloutScalingWhenPaused (5.79s) PASS test/e2e.TestCanarySuite/TestRolloutWithMaxSurgeScalingDuringUpdate (3.92s) PASS test/e2e.TestExperimentSuite/TestExperimentWithMeasurementRetentionMetrics (36.82s) PASS test/e2e.TestExperimentSuite/TestExperimentWithMultiportServiceAndScaleDownDelay (18.09s) PASS test/e2e.TestExperimentSuite/TestExperimentWithServiceAndScaleDownDelay (17.97s) PASS test/e2e.TestExperimentSuite/TestExperimentWithServiceNameAndScaleDownDelay (17.99s) PASS test/e2e.TestExperimentSuite/TestRolloutWithExperimentAndAnalysis (10.14s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenExceedProgressDeadlineAbort (10.24s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenPreviewReplicaCountPromoteFull (8.46s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenPromoteFull (9.44s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownDelay (9.95s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownOnAbort (9.87s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenToCanary (4.93s) PASS test/e2e.TestFunctionalSuite/TestBlueGreenUpdate (6.25s) PASS test/e2e.TestFunctionalSuite/TestCanaryPromoteFull (9.67s) PASS test/e2e.TestFunctionalSuite/TestCompleteRolloutRestart (22.04s) PASS test/e2e.TestFunctionalSuite/TestContainerResourceFormats (3.08s) PASS test/e2e.TestFunctionalSuite/TestFixInvalidSpec (22.03s) PASS test/e2e.TestFunctionalSuite/TestKubectlWaitForCompleted (24.18s) PASS test/e2e.TestFunctionalSuite/TestKubectlWaitForPaused (7.75s) PASS test/e2e.TestFunctionalSuite/TestMalformedRollout (6.41s) PASS test/e2e.TestFunctionalSuite/TestMalformedRolloutEphemeralCtr (3.02s) PASS test/e2e.TestFunctionalSuite/TestNeverScaleDown (6.33s) PASS test/e2e.TestFunctionalSuite/TestRolloutAbortRetryPromote (8.79s) PASS test/e2e.TestFunctionalSuite/TestRolloutBadToBadUpdate (23.95s) PASS test/e2e.TestFunctionalSuite/TestRolloutBadToGoodSetWeightUpdate (22.18s) PASS test/e2e.TestFunctionalSuite/TestRolloutBadToGoodUpdate (17.05s) PASS test/e2e.TestFunctionalSuite/TestRolloutGoodToBadUpdate (24.80s) PASS test/e2e.TestFunctionalSuite/TestRolloutPDBRestart (46.47s) PASS test/e2e.TestFunctionalSuite/TestRolloutPauseDurationGreaterThanProgressDeadlineSeconds (36.17s) PASS test/e2e.TestFunctionalSuite/TestRolloutPodVolumesItemsSpec (9.09s) PASS test/e2e.TestFunctionalSuite/TestRolloutRestart (18.80s) PASS test/e2e.TestFunctionalSuite/TestScaleDownOnSuccess (5.22s) PASS test/e2e.TestFunctionalSuite/TestScaleDownProgressively (5.02s) PASS test/e2e.TestFunctionalSuite/TestWorkloadRef (17.22s) PASS test/e2e.TestFunctionalSuite/TestWorkloadRefTemplate (2.79s) PASS test/e2e.TestHeaderRoutingSuite (14.12s) PASS test/e2e.TestHeaderRoutingSuite/TestIstioHostHeaderRoute (13.71s) PASS test/e2e.TestIstioSuite (130.31s) PASS test/e2e.TestIstioSuite/TestIstioAbortUpdate (7.77s) PASS test/e2e.TestIstioSuite/TestIstioAbortUpdateDeleteAllCanaryPods (8.13s) PASS test/e2e.TestIstioSuite/TestIstioHostSplit (17.20s) PASS test/e2e.TestIstioSuite/TestIstioHostSplitExperimentStep (19.04s) PASS test/e2e.TestIstioSuite/TestIstioHostSplitOnlyTls (9.12s) PASS test/e2e.TestIstioSuite/TestIstioPingPongUpdate (12.48s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplit (11.16s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplitExperimentStep (19.31s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplitInStableDownscaleAfterCanaryAbort (8.05s) PASS test/e2e.TestIstioSuite/TestIstioSubsetSplitSingleRoute (8.71s) PASS test/e2e.TestIstioSuite/TestIstioUpdateInMiddleZeroCanaryReplicas (8.89s) PASS test/e2e.TestMirrorRouteSuite (14.09s) PASS test/e2e.TestMirrorRouteSuite/TestIstioHostMirrorRoute (13.68s) PASS test/e2e.TestRollbackSuite (57.92s) PASS test/e2e.TestRollbackSuite/TestRollbackAnalysisOutsideWindow (32.13s) PASS test/e2e.TestRollbackSuite/TestRollbackAnalysisWithinWindow (23.08s) These were the tests that failed: --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.24s) --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.27s) --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.36s) --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.37s) --- FAIL: TestAPISIXSuite/TestAPISIXCanarySetHeaderStep (5.58s) --- FAIL: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (19.10s) --- FAIL: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (22.12s) --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.02s) --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.61s) --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.64s) --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (36.70s) --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (37.66s) --- FAIL: TestExperimentSuite/TestExperimentWithDryRunMetrics (37.67s) --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.73s) --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.74s) --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.75s) --- FAIL: TestFunctionalSuite/TestControllerMetrics (0.76s) E2E tests file is complete: DONE 6 runs, 145 tests, 6 skipped, 40 failures in 3034.276s * SUCCESS: No unexpected errors occurred. * Cleaning up hack/run-upstream-argo-rollouts-e2e-tests.sh: line 7: killall: command not found hack/run-upstream-argo-rollouts-e2e-tests.sh: line 8: killall: command not found + [[ 0 != 0 ]] + kubectl delete ns argo-rollouts namespace "argo-rollouts" deleted + kubectl wait --timeout=5m --for=delete namespace/argo-rollouts + kubectl create ns argo-rollouts namespace/argo-rollouts created + kubectl config set-context --current --namespace=argo-rollouts Context "default/api-nasrosa-415-ddru-p1-openshiftapps-com:6443/cluster-admin" modified. + cat + kubectl apply -f /tmp/tmp.ofaHpIuBVk/rollout-manager.yaml rolloutmanager.argoproj.io/argo-rollout created + cd /tmp/tmp.ofaHpIuBVk + git clone https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-openshift Cloning into 'rollouts-plugin-trafficrouter-openshift'... + cd /tmp/tmp.ofaHpIuBVk/rollouts-plugin-trafficrouter-openshift + git checkout 8b4125a7f9ecffb0247df91a4c890f88c0c523b7 Note: switching to '8b4125a7f9ecffb0247df91a4c890f88c0c523b7'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 8b4125a Switch lint to use Makefile, fix go.mod + make test-e2e go test -v -p=1 -timeout=20m -race -count=1 -coverprofile=coverage.out ./tests/e2e go: downloading go1.22.2 (linux/amd64) go: downloading github.com/argoproj/argo-rollouts v1.6.6 go: downloading github.com/onsi/gomega v1.33.1 go: downloading github.com/onsi/ginkgo/v2 v2.19.0 go: downloading k8s.io/client-go v0.26.3 go: downloading github.com/openshift/api v0.0.0-20230417092139-1b2161d23365 go: downloading k8s.io/apimachinery v0.26.3 go: downloading github.com/openshift/client-go v0.0.0-20230419131419-497c7032c581 go: downloading k8s.io/api v0.26.3 go: downloading github.com/go-logr/logr v1.4.1 go: downloading golang.org/x/net v0.25.0 go: downloading golang.org/x/sys v0.20.0 go: downloading golang.org/x/text v0.15.0 go: downloading github.com/spf13/cobra v1.7.0 go: downloading k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 go: downloading github.com/hashicorp/go-plugin v1.4.10 go: downloading k8s.io/cli-runtime v0.26.3 go: downloading k8s.io/kubernetes v1.26.3 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading google.golang.org/grpc v1.57.1 go: downloading github.com/bombsimon/logrusr/v4 v4.0.0 go: downloading golang.org/x/term v0.20.0 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading k8s.io/component-base v0.26.3 go: downloading k8s.io/apiserver v0.26.3 go: downloading k8s.io/component-helpers v0.26.3 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.42.0 === RUN TestPlugin Running Suite: Plugin e2e Tests Suite - /tmp/tmp.ofaHpIuBVk/rollouts-plugin-trafficrouter-openshift/tests/e2e ============================================================================================================= Random Seed: 1733294495 Will run 3 of 3 specs service/rollouts-demo-stable created service/rollouts-demo-canary created route.route.openshift.io/rollouts-demo created rollout.argoproj.io/rollouts-demo created Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy W1204 06:41:40.249118 89266 warnings.go:70] unknown field "spec.template.metadata.creationTimestamp" Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy •service/rollouts-demo-stable created service/rollouts-demo-canary created route.route.openshift.io/rollouts-demo-prod created route.route.openshift.io/rollouts-demo-stage created rollout.argoproj.io/rollouts-demo created Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy W1204 06:42:05.771029 89266 warnings.go:70] unknown field "spec.template.metadata.creationTimestamp" Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy •service/rollouts-demo-stable created service/rollouts-demo-canary created route.route.openshift.io/rollouts-demo created analysistemplate.argoproj.io/sample-template created rollout.argoproj.io/rollouts-demo created Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy W1204 06:42:41.246661 89266 warnings.go:70] unknown field "spec.template.metadata.creationTimestamp" Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Progressing, want Paused Rollout phase mismatch, got Paused, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy Rollout phase mismatch, got Progressing, want Healthy • Ran 3 of 3 Specs in 83.561 seconds SUCCESS! -- 3 Passed | 0 Failed | 0 Pending | 0 Skipped --- PASS: TestPlugin (83.56s) PASS coverage: [no statements] ok github.com/argoproj-labs/rollouts-plugin-trafficrouter-openshift/tests/e2e 84.665s coverage: [no statements] + [[ 0 != 0 ]] + oc patch subscription openshift-gitops-operator -n openshift-operators --type json --patch '[{"op": "remove", "path": "/spec/config"}]' subscription.operators.coreos.com/openshift-gitops-operator patched + update_result + [[ 2 != 0 ]] + echo test-argo-rollouts + tee /install-dir/nasrosa-415-result.txt test-argo-rollouts + exit 0 undefined step-run-kuttl-e2e-tests $RUN_KUTTL_TESTS is set to false, skipping the execution 0 http://uploader.devtools-gitops--pipeline.svc.cluster.local:8080/CI/z-stream-test-pipeline/241204-052308/ undefined step-run-argocd-e2e-tests $RUN_ARGOCD_E2E_TESTS is set to false, skipping the execution 0 http://uploader.devtools-gitops--pipeline.svc.cluster.local:8080/CI/z-stream-test-pipeline/241204-052308/ undefined step-evaluate-failed-tasks Argo rollouts test execution has failed with exit code 2 Refer to test-argo-rollouts task undefined step-send-slack-notification ok