-
Story
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
False
-
False
-
Undefined
-
-
0
-
0
As a Network Edge team member,
I want to know which tests are not run,
so that I know which features lack coverage and which tests need attention.
Network Edge components have unit and end-to-end tests in several locations. Some tests are in the repository for the component under test (for example, ingress operator-related tests are in the openshift/cluster-ingress-operator repository), and some tests are in the openshift/origin repository (in particular, all end-to-end tests related to the router, idling, and ingress are in this repository).
Tests for which Network Edge is responsible include unit and end-to-end tests in the following repositories:
- https://github.com/openshift/images/ (specifically tests under egress/ and ipfailover/).
- https://github.com/openshift/openshift-controller-manager/ (specifically tests under pkg/unidling/controller/ and pkg/route/ingress/).
- https://github.com/openshift/origin/ (specifically tests under test/extended/idling/ and test/extended/router/, and some tests under test/extended/testdata/cmd/test/cmd/, test/e2e/upgrade/service/, and vendor/k8s.io/kubernetes/test/e2e/network).
Several mechanisms are used to cause tests not to run:
- Adding t.Skip(...) statements in tests that use Go's testing package, or g.Skip(...) in tests that use Ginkgo.
- Commenting out test code using # in Shell-based tests in openshift/origin.
- Explicitly listing test names in https://github.com/openshift/origin/blob/master/vendor/k8s.io/kubernetes/openshift-hack/e2e/annotate/rules.go.
- Adding tags such as "[Disabled:Broken]" or "[Local]" in Ginkgo contexts (see ExcludedTests in rules.go).
- Listing test names in the openshift/release CI definitions (see for example NE-534 and https://github.com/openshift/release/blob/52c4f09611d48ace8a36f309841faf670e0287ff/ci-operator/step-registry/openshift/e2e/aws/proxy/openshift-e2e-aws-proxy-workflow.yaml#L22-L25).
We need to document which tests exist but are not run. We should track these tests in Jira so that we can prioritize re-enabling them.