[2047012926] Upstream Reporter: Adam Kaplan
Upstream issue status: Open
Upstream description:

Is there an existing feature request for this?

• ☒ I have searched the existing feature requests

Is your feature request related to a problem or use-case? Please describe.

The current e2e test suite for shipwright/build contains tests for multiple features, however it can only be run as an "all or nothing" approach. We should be able to annotate/label tests to identify the associated feature and run subsets of e2e tests.

Use case: Downstream distributions may not support all the features in the "upstream". The build repo also tests the samples, which are not guaranteed to work on all Kubernetes distributions (OpenShift in particular has skews around security that can make these tests require additional setup).

Describe the solution that you would like.

Ginkgo v2 has "primary" support for labels in spec descriptors [1]. These can be used to narrow tests. We can use labels like [FEATURE: xxx] to narrow features, and others like [CORE] to identify specs that test the most core/stable features.

[1] https://onsi.github.io/ginkgo/#spec-labels

Describe alternatives you have considered.

We could leave descriptions in place and add description-based filtering [1].

[1] https://onsi.github.io/ginkgo/#description-based-filtering

Anything else?

The ginkgo docs recommend using the ginkgo cli to run tests and pass in the right flags to do this kind of filtering.

We might be able to keep the existing go test -based approach by passing in flags via -ginkgo.nnn [1]

[1] https://github.com/onsi/ginkgo/issues/381