[GITOPS-5939] [Test Execution] - ArgoCD Rollouts E2E Tests | AWS - 4.15 | ARM - Red Hat Issue Tracker

Type: Task
Resolution: Done
Priority: Undefined
Fix Version/s: 1.15.0
Affects Version/s: None
Component/s: Testing
Labels:
None

Epic Link:
GitOps 1.15.0 Release Test Execution
Blocked:
False
Blocked Reason:
None
Ready:
False
Intelligence Requested:
Market:

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

The run-rollouts-e2e-tests.sh shell script within gitops-operator repo can be used to automatically run the Argo Rollouts operator E2E tests, the upstream Argo Rollouts tests, and the Argo Rollouts OpenShift Routes traffic management plugin E2E tests.The test and test configurations that run are:

Operator E2E tests for cluster-scoped Argo Rollouts installs (tests are defined here: https://github.com/argoproj-labs/argo-rollouts-manager/tree/main/tests/e2e)
The upstream 'argoproj/argo-rollouts' E2E tests, running against cluster-scoped Argo Rollouts installed va Operator (tests are defined here: https://github.com/argoproj/argo-rollouts/tree/master/test/e2e)
OpenShift Route Plugin E2E tests on cluster-scoped Argo Rollouts (tests are defined here: https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-openshift/tree/main/tests/e2e)

is cloned by

GITOPS-6565 [Test Execution] - ArgoCD Rollouts E2E Tests | AWS - 4.15 | ARM

Closed

Julia Teslia added a comment - 2024/12/11 10:47 AM - edited

ArgoCD Rollouts E2E tests:

PASS:

awk '/PASS/ {print $2}' /home/jteslia/Downloads/rollouts.log | sort -u
test/e2e.TestAnalysisSuite
test/e2e.TestAnalysisSuite/TestAnalysisWithArgs
test/e2e.TestAnalysisSuite/TestAnalysisWithSecret
test/e2e.TestAnalysisSuite/TestBackgroundAnalysisWithArgs
test/e2e.TestAnalysisSuite/TestBlueGreenAbortAndUpdate
test/e2e.TestAnalysisSuite/TestBlueGreenAnalysis
test/e2e.TestAnalysisSuite/TestBlueGreenKitchenSink
test/e2e.TestAnalysisSuite/TestBlueGreenPostPromotionFail
test/e2e.TestAnalysisSuite/TestBlueGreenPrePromotionFail
test/e2e.TestAnalysisSuite/TestCanaryBackgroundAnalysis
test/e2e.TestAnalysisSuite/TestCanaryFailInlineMultipleAnalysis
test/e2e.TestAnalysisSuite/TestCanaryInlineAnalysis
test/e2e.TestAnalysisSuite/TestCanaryInlineMultipleAnalysis
test/e2e.TestAnalysisSuite/TestMultipleAnalysis
test/e2e.TestAPISIXSuite/TestAPISIXCanaryStep
test/e2e.TestAWSSuite
test/e2e.TestAWSSuite/TestALBExperimentStep
test/e2e.TestAWSSuite/TestALBExperimentStepMultiIngress
test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeight
test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress
test/e2e.TestAWSSuite/TestAlbHeaderRoute
test/e2e.TestAWSSuite/TestAlbHeaderRouteMultiIngress
test/e2e.TestAWSSuite/TestALBMesh_PingPong_StableCanary_Update
test/e2e.TestAWSSuite/TestALBPingPongUpdate
test/e2e.TestAWSSuite/TestALBPingPongUpdateMultiIngress
test/e2e.TestBlueGreenSuite
test/e2e.TestBlueGreenSuite/TestBlueGreenPreviewReplicaCount
test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithoutPause
test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithPause
test/e2e.TestBlueGreenSuite/TestEphemeralMetadata
test/e2e.TestCanarySuite
test/e2e.TestCanarySuite/TestCanaryDynamicStableScale
test/e2e.TestCanarySuite/TestCanaryDynamicStableScaleRollbackToStable
test/e2e.TestCanarySuite/TestCanaryProgressDeadlineExceededWithPause
test/e2e.TestCanarySuite/TestCanaryScaleDownDelay
test/e2e.TestCanarySuite/TestCanaryScaleDownDelayWithProgressDeadline
test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbort
test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbortNoTrafficRouting
test/e2e.TestCanarySuite/TestCanarySetCanaryScale
test/e2e.TestCanarySuite/TestCanaryUnScaleDownOnAbort
test/e2e.TestCanarySuite/TestCanaryWithPausedRollout
test/e2e.TestCanarySuite/TestEphemeralMetadata
test/e2e.TestCanarySuite/TestReduceWeightAndHonorMaxUnavailable
test/e2e.TestCanarySuite/TestRolloutScalingDuringUpdate
test/e2e.TestCanarySuite/TestRolloutScalingWhenPaused
test/e2e.TestCanarySuite/TestRolloutWithMaxSurgeScalingDuringUpdate
test/e2e.TestExperimentSuite/TestExperimentWithMeasurementRetentionMetrics
test/e2e.TestExperimentSuite/TestExperimentWithMultiportServiceAndScaleDownDelay
test/e2e.TestExperimentSuite/TestExperimentWithServiceAndScaleDownDelay
test/e2e.TestExperimentSuite/TestExperimentWithServiceNameAndScaleDownDelay
test/e2e.TestExperimentSuite/TestRolloutWithExperimentAndAnalysis
test/e2e.TestFunctionalSuite/TestBlueGreenExceedProgressDeadlineAbort
test/e2e.TestFunctionalSuite/TestBlueGreenPreviewReplicaCountPromoteFull
test/e2e.TestFunctionalSuite/TestBlueGreenPromoteFull
test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownDelay
test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownOnAbort
test/e2e.TestFunctionalSuite/TestBlueGreenToCanary
test/e2e.TestFunctionalSuite/TestBlueGreenUpdate
test/e2e.TestFunctionalSuite/TestCanaryPromoteFull
test/e2e.TestFunctionalSuite/TestCompleteRolloutRestart
test/e2e.TestFunctionalSuite/TestContainerResourceFormats
test/e2e.TestFunctionalSuite/TestFixInvalidSpec
test/e2e.TestFunctionalSuite/TestKubectlWaitForCompleted
test/e2e.TestFunctionalSuite/TestKubectlWaitForPaused
test/e2e.TestFunctionalSuite/TestMalformedRollout
test/e2e.TestFunctionalSuite/TestMalformedRolloutEphemeralCtr
test/e2e.TestFunctionalSuite/TestNeverScaleDown
test/e2e.TestFunctionalSuite/TestRolloutAbortRetryPromote
test/e2e.TestFunctionalSuite/TestRolloutBadToBadUpdate
test/e2e.TestFunctionalSuite/TestRolloutBadToGoodSetWeightUpdate
test/e2e.TestFunctionalSuite/TestRolloutBadToGoodUpdate
test/e2e.TestFunctionalSuite/TestRolloutGoodToBadUpdate
test/e2e.TestFunctionalSuite/TestRolloutPauseDurationGreaterThanProgressDeadlineSeconds
test/e2e.TestFunctionalSuite/TestRolloutPDBRestart
test/e2e.TestFunctionalSuite/TestRolloutPodVolumesItemsSpec
test/e2e.TestFunctionalSuite/TestRolloutRestart
test/e2e.TestFunctionalSuite/TestScaleDownOnSuccess
test/e2e.TestFunctionalSuite/TestScaleDownProgressively
test/e2e.TestFunctionalSuite/TestWorkloadRef
test/e2e.TestFunctionalSuite/TestWorkloadRefTemplate
test/e2e.TestHeaderRoutingSuite
test/e2e.TestHeaderRoutingSuite/TestIstioHostHeaderRoute
test/e2e.TestIstioSuite
test/e2e.TestIstioSuite/TestIstioAbortUpdate
test/e2e.TestIstioSuite/TestIstioAbortUpdateDeleteAllCanaryPods
test/e2e.TestIstioSuite/TestIstioHostSplit
test/e2e.TestIstioSuite/TestIstioHostSplitExperimentStep
test/e2e.TestIstioSuite/TestIstioHostSplitOnlyTls
test/e2e.TestIstioSuite/TestIstioPingPongUpdate
test/e2e.TestIstioSuite/TestIstioSubsetSplit
test/e2e.TestIstioSuite/TestIstioSubsetSplitExperimentStep
test/e2e.TestIstioSuite/TestIstioSubsetSplitInStableDownscaleAfterCanaryAbort
test/e2e.TestIstioSuite/TestIstioSubsetSplitSingleRoute
test/e2e.TestIstioSuite/TestIstioUpdateInMiddleZeroCanaryReplicas
test/e2e.TestMirrorRouteSuite
test/e2e.TestMirrorRouteSuite/TestIstioHostMirrorRoute
test/e2e.TestRollbackSuite
test/e2e.TestRollbackSuite/TestRollbackAnalysisOutsideWindow
test/e2e.TestRollbackSuite/TestRollbackAnalysisWithinWindow

FAIL:

awk '/FAIL/ {print $3}' /home/jteslia/Downloads/rollouts.log | sort -u

TestAPISIXSuite/TestAPISIXCanarySetHeaderStep
TestCanarySuite/TestCanaryScaleDownOnAbort
TestExperimentSuite/TestExperimentWithDryRunMetrics
TestFunctionalSuite/TestControllerMetrics

Julia Teslia added a comment - 2024/12/11 10:47 AM - edited ArgoCD Rollouts E2E tests: PASS: awk '/PASS/ {print $2}' /home/jteslia/Downloads/rollouts.log | sort -u test/e2e.TestAnalysisSuite test/e2e.TestAnalysisSuite/TestAnalysisWithArgs test/e2e.TestAnalysisSuite/TestAnalysisWithSecret test/e2e.TestAnalysisSuite/TestBackgroundAnalysisWithArgs test/e2e.TestAnalysisSuite/TestBlueGreenAbortAndUpdate test/e2e.TestAnalysisSuite/TestBlueGreenAnalysis test/e2e.TestAnalysisSuite/TestBlueGreenKitchenSink test/e2e.TestAnalysisSuite/TestBlueGreenPostPromotionFail test/e2e.TestAnalysisSuite/TestBlueGreenPrePromotionFail test/e2e.TestAnalysisSuite/TestCanaryBackgroundAnalysis test/e2e.TestAnalysisSuite/TestCanaryFailInlineMultipleAnalysis test/e2e.TestAnalysisSuite/TestCanaryInlineAnalysis test/e2e.TestAnalysisSuite/TestCanaryInlineMultipleAnalysis test/e2e.TestAnalysisSuite/TestMultipleAnalysis test/e2e.TestAPISIXSuite/TestAPISIXCanaryStep test/e2e.TestAWSSuite test/e2e.TestAWSSuite/TestALBExperimentStep test/e2e.TestAWSSuite/TestALBExperimentStepMultiIngress test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeight test/e2e.TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress test/e2e.TestAWSSuite/TestAlbHeaderRoute test/e2e.TestAWSSuite/TestAlbHeaderRouteMultiIngress test/e2e.TestAWSSuite/TestALBMesh_PingPong_StableCanary_Update test/e2e.TestAWSSuite/TestALBPingPongUpdate test/e2e.TestAWSSuite/TestALBPingPongUpdateMultiIngress test/e2e.TestBlueGreenSuite test/e2e.TestBlueGreenSuite/TestBlueGreenPreviewReplicaCount test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithoutPause test/e2e.TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithPause test/e2e.TestBlueGreenSuite/TestEphemeralMetadata test/e2e.TestCanarySuite test/e2e.TestCanarySuite/TestCanaryDynamicStableScale test/e2e.TestCanarySuite/TestCanaryDynamicStableScaleRollbackToStable test/e2e.TestCanarySuite/TestCanaryProgressDeadlineExceededWithPause test/e2e.TestCanarySuite/TestCanaryScaleDownDelay test/e2e.TestCanarySuite/TestCanaryScaleDownDelayWithProgressDeadline test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbort test/e2e.TestCanarySuite/TestCanaryScaleDownOnAbortNoTrafficRouting test/e2e.TestCanarySuite/TestCanarySetCanaryScale test/e2e.TestCanarySuite/TestCanaryUnScaleDownOnAbort test/e2e.TestCanarySuite/TestCanaryWithPausedRollout test/e2e.TestCanarySuite/TestEphemeralMetadata test/e2e.TestCanarySuite/TestReduceWeightAndHonorMaxUnavailable test/e2e.TestCanarySuite/TestRolloutScalingDuringUpdate test/e2e.TestCanarySuite/TestRolloutScalingWhenPaused test/e2e.TestCanarySuite/TestRolloutWithMaxSurgeScalingDuringUpdate test/e2e.TestExperimentSuite/TestExperimentWithMeasurementRetentionMetrics test/e2e.TestExperimentSuite/TestExperimentWithMultiportServiceAndScaleDownDelay test/e2e.TestExperimentSuite/TestExperimentWithServiceAndScaleDownDelay test/e2e.TestExperimentSuite/TestExperimentWithServiceNameAndScaleDownDelay test/e2e.TestExperimentSuite/TestRolloutWithExperimentAndAnalysis test/e2e.TestFunctionalSuite/TestBlueGreenExceedProgressDeadlineAbort test/e2e.TestFunctionalSuite/TestBlueGreenPreviewReplicaCountPromoteFull test/e2e.TestFunctionalSuite/TestBlueGreenPromoteFull test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownDelay test/e2e.TestFunctionalSuite/TestBlueGreenScaleDownOnAbort test/e2e.TestFunctionalSuite/TestBlueGreenToCanary test/e2e.TestFunctionalSuite/TestBlueGreenUpdate test/e2e.TestFunctionalSuite/TestCanaryPromoteFull test/e2e.TestFunctionalSuite/TestCompleteRolloutRestart test/e2e.TestFunctionalSuite/TestContainerResourceFormats test/e2e.TestFunctionalSuite/TestFixInvalidSpec test/e2e.TestFunctionalSuite/TestKubectlWaitForCompleted test/e2e.TestFunctionalSuite/TestKubectlWaitForPaused test/e2e.TestFunctionalSuite/TestMalformedRollout test/e2e.TestFunctionalSuite/TestMalformedRolloutEphemeralCtr test/e2e.TestFunctionalSuite/TestNeverScaleDown test/e2e.TestFunctionalSuite/TestRolloutAbortRetryPromote test/e2e.TestFunctionalSuite/TestRolloutBadToBadUpdate test/e2e.TestFunctionalSuite/TestRolloutBadToGoodSetWeightUpdate test/e2e.TestFunctionalSuite/TestRolloutBadToGoodUpdate test/e2e.TestFunctionalSuite/TestRolloutGoodToBadUpdate test/e2e.TestFunctionalSuite/TestRolloutPauseDurationGreaterThanProgressDeadlineSeconds test/e2e.TestFunctionalSuite/TestRolloutPDBRestart test/e2e.TestFunctionalSuite/TestRolloutPodVolumesItemsSpec test/e2e.TestFunctionalSuite/TestRolloutRestart test/e2e.TestFunctionalSuite/TestScaleDownOnSuccess test/e2e.TestFunctionalSuite/TestScaleDownProgressively test/e2e.TestFunctionalSuite/TestWorkloadRef test/e2e.TestFunctionalSuite/TestWorkloadRefTemplate test/e2e.TestHeaderRoutingSuite test/e2e.TestHeaderRoutingSuite/TestIstioHostHeaderRoute test/e2e.TestIstioSuite test/e2e.TestIstioSuite/TestIstioAbortUpdate test/e2e.TestIstioSuite/TestIstioAbortUpdateDeleteAllCanaryPods test/e2e.TestIstioSuite/TestIstioHostSplit test/e2e.TestIstioSuite/TestIstioHostSplitExperimentStep test/e2e.TestIstioSuite/TestIstioHostSplitOnlyTls test/e2e.TestIstioSuite/TestIstioPingPongUpdate test/e2e.TestIstioSuite/TestIstioSubsetSplit test/e2e.TestIstioSuite/TestIstioSubsetSplitExperimentStep test/e2e.TestIstioSuite/TestIstioSubsetSplitInStableDownscaleAfterCanaryAbort test/e2e.TestIstioSuite/TestIstioSubsetSplitSingleRoute test/e2e.TestIstioSuite/TestIstioUpdateInMiddleZeroCanaryReplicas test/e2e.TestMirrorRouteSuite test/e2e.TestMirrorRouteSuite/TestIstioHostMirrorRoute test/e2e.TestRollbackSuite test/e2e.TestRollbackSuite/TestRollbackAnalysisOutsideWindow test/e2e.TestRollbackSuite/TestRollbackAnalysisWithinWindow FAIL: awk '/FAIL/ {print $3}' /home/jteslia/Downloads/rollouts.log | sort -u TestAPISIXSuite/TestAPISIXCanarySetHeaderStep TestCanarySuite/TestCanaryScaleDownOnAbort TestExperimentSuite/TestExperimentWithDryRunMetrics TestFunctionalSuite/TestControllerMetrics

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

[GITOPS-5939] [Test Execution] - ArgoCD Rollouts E2E Tests | AWS - 4.15 | ARM

Collapse comment: Julia Teslia added a comment - 2024/12/11 10:47 AM, Edited by Julia Teslia - 2024/12/11 10:48 AM

Expand comment: Julia Teslia added a comment - 2024/12/11 10:47 AM, Edited by Julia Teslia - 2024/12/11 10:48 AM

People

Dates