Story (Required)

The acceptance tests should include the following Shipwright features:

Simple Build: The basic build functionality of Shipwright should be tested. This involves creating a build configuration, specifying the source code repository, and building a container image using the below.

• Buildpacks: Shipwright's buildpack support needs to be validated. This includes testing the ability to automatically detect and apply appropriate buildpacks based on the source code.

• Source-to-Image (S2I): QE should verify Shipwright's integration with Source-to-Image. This involves creating a build configuration using S2I and ensuring that the resulting image is generated correctly.

• Buildah: The integration of the Buildah feature needs to be thoroughly tested. This includes testing the Buildah workflow, verifying its compatibility with shipwright’s existing features and ensuring that Buildah generates images that are built correctly within the shipwright environment.

  Background (Required)

A release acceptance test needs to be triggered to ensure that the integration between Shipwright and OpenShift is functioning as expected. This test will encompass various aspects of Shipwright's functionality and its integration within the OpenShift ecosystem.

The acceptance test process includes the following steps:

Operator Installation: The QE team will validate if the Shipwright operator is correctly installed on the OpenShift cluster. This involves ensuring that the installation process is successful and the operator is deployed as expected.

Operator Visibility: In addition to installation, the QE team will verify that the Shipwright operator is visible and accessible on the OperatorHub within the OpenShift cluster's console. This ensures that the operator is discoverable and can be easily managed by users.

Integration Testing: The acceptance tests will cover the integration of Shipwright with various build strategies, including simple builds, buildpacks, S2I, and the Buildah feature. The tests will assess the seamless collaboration between Shipwright and these strategies, ensuring that the build process is accurate and reliable.

Stability and Performance: The acceptance tests will also include stability and performance assessments. This ensures that Shipwright can handle different workloads and scenarios without encountering issues.

Error Handling: The QE team will intentionally introduce errors or unexpected situations to evaluate how Shipwright handles such scenarios. Error messages, logging, and recovery mechanisms will be analyzed for accuracy and effectiveness.

The QE team will document and report any issues, discrepancies, or observations encountered during the release acceptance test. This information will be critical for further refinement and improvement of Shipwright's integration within the OpenShift ecosystem.

The acceptance test repository can be found at: Acceptance Test Repository

Out of scope

It has been identified that the operator will not automatically install cluster build strategies. Therefore, manual installation of these build strategies is necessary to facilitate testing. QE will perform manual installations following the documented procedures. The curated build strategies will be hosted at https://github.com/redhat-developer/openshift-builds-catalog

Acceptance Criteria (Mandatory)

The acceptance test process includes the following steps:

Operator Installation: The QE team will validate if the Shipwright operator is correctly installed on the OpenShift cluster. This involves ensuring that the installation process is successful and the operator is deployed as expected.

Operator Visibility: In addition to installation, the QE team will verify that the Shipwright operator is visible and accessible on the OperatorHub within the OpenShift cluster's console. This ensures that the operator is discoverable and can be easily managed by users.

Integration Testing: The acceptance tests will cover the integration of Shipwright with various build strategies, including simple builds, buildpacks, S2I, and the Buildah feature. The tests will assess the seamless collaboration between Shipwright and these strategies, ensuring that the build process is accurate and reliable.

Stability and Performance: The acceptance tests will also include stability and performance assessments. This ensures that Shipwright can handle different workloads and scenarios without encountering issues.

Done Checklist

• Code is completed, reviewed, documented and checked in
• Unit and integration test automation have been delivered and running cleanly in continuous integration/staging/canary environment
• Continuous Delivery pipeline(s) is able to proceed with new code included
• Customer facing documentation, API docs etc. are produced/updated, reviewed and published
• Acceptance criteria are met