Uploaded image for project: 'OpenShift GitOps'
  1. OpenShift GitOps
  2. GITOPS-7335

[ImageUpdater][CRD] End-to-End (E2E) Testing and Validation

XMLWordPrintable

    • Icon: Epic Epic
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • None
    • ImageUpdater
    • [ImageUpdater][CRD] End-to-End (E2E) Testing and Validation
    • False
    • Hide

      None

      Show
      None
    • False
    • In Progress
    • SECFLOWOTL-158 - Image Updater: Refactor update cycle logic and introduce Custom Resource Definition
    • 20% To Do, 20% In Progress, 60% Done

      Epic Goal

       

      To develop and implement a suite of end-to-end (e2e) tests that validate the complete, real-world functionality of the ImageUpdater controller. This involves deploying the controller and its dependencies into a live Kubernetes cluster and verifying that it correctly manages Argo CD Applications based on ImageUpdater CRs.

       

      Why is this important?

       

      While unit and integration tests are essential for verifying individual components, they cannot fully capture the complex interactions between the controller, the Kubernetes API server, Argo CD, and external Git repositories. E2E tests are the final and most important validation step to ensure that all the pieces work together correctly.

      This epic is critical for:

      • Building Confidence: Proving that the entire feature works from start to finish.
      • Preventing Regressions: Creating an automated safety net that catches bugs before they reach production.
      • Validating Real-World Scenarios: Testing complex user workflows that are difficult to simulate in unit tests, such as Git write-back operations and interactions with a live Argo CD instance.

       

      Scenarios

       

      • E2E Test for Helm Update with Git Write-Back:
        • An ImageUpdater CR is created to watch an image used in a Helm-based Application. A new version of the image is pushed to a registry. The e2e test verifies that the controller correctly clones the Git repository, updates the image.tag in the values.yaml file, and commits and pushes the change.
      • E2E Test for Kustomize Update with Direct argocd Write-Back:
        • An ImageUpdater CR with writeBackMethod: argocd is created. The e2e test verifies that the controller directly updates the Application CR's spec to use a new Kustomize image tag.
      • E2E Test for Conflict Resolution (Webhook):
        • The test creates an ImageUpdater CR that manages app-A. It then attempts to create a second, conflicting CR that also targets app-A. The test verifies that the validating admission webhook correctly rejects the second CR with the expected error message.

       

      Other Considerations

       

      • Out of Scope: This epic focuses on creating the e2e test suite and the necessary test infrastructure. It does not cover performance or scale testing.
      • Dependencies: These tests require a running Kubernetes/OpenShift cluster with Argo CD and cert-manager (for the webhook) installed. The tests will also depend on the controller's deployment manifests and a container image being available.
      • Previous Works: The project has an existing e2e test framework that can likely be adapted and extended for the new CRD-based controller.
      • Unanswered Questions: What test infrastructure (e.g., dedicated cluster, KinD) will be used to run these tests in the CI/CD pipeline?

      Definition of Ready

      • The epic has been broken down into stories.
      • Stories have been scoped.
      • The epic has been stack ranked.

      Definition of Done

      • Code Complete:
        • All code has been written, reviewed, and approved.
      • Tested:
        • Unit tests have been written and passed.
        • Integration tests have been completed.
        • System tests have been conducted, and all critical bugs have been fixed.
        • Tested on OpenShift either upstream or downstream on a local build.
      • Documentation:
        • User documentation or release notes have been written.
      • Build:
        • Code has been successfully built and integrated into the main repository / project.
      • Review:
        • Code has been peer-reviewed and meets coding standards.
        • All acceptance criteria defined in the user story have been met.
        • Tested by reviewer on OpenShift.
      • Deployment:
        • The feature has been deployed on OpenShift cluster for testing.
      • Acceptance:
        • Product Manager or stakeholder has reviewed and accepted the work.

              dkarpele@redhat.com Denis Karpelevich
              dkarpele@redhat.com Denis Karpelevich
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: