The MODH service and all other services offered within MODH (Red Hat and external ISV services) need to provide a test harness suite that can plug into the OSD e2e testing. When the test suites are run, we need to get data on the outcome of the tests and display metrics so the appropriate teams can consume the results.

Requirements:

1. Each ISV service in MODH must have a basic or higher level test suite that adheres to the following parameters:

• The test suite must be packaged as an image and should be self-contained.
• The test suite must produce a “junit-<operator_name>.xml” that contains test names, pass/fails, and the time the test segment took in the /test-run-results directory. The XSD file can be found here and an example junit.xml operator test run result can be found here.
• The test suite must produce metadata to addon-metadata.json in the /test-run-results.

2. The test suites created based on #1 above must be packaged in a bundle according to OSD specifications and the bundle must reside in the OSD managed GitLab repository.

3. Any alerts from the execution of test suites (eg. alert of a failure) must be routed to the appropriate ISV contact for the service.

4. The system must support the display of metrics for the results of test suite execution jobs for all MODH test suites including the MODH test suite and ISV test suites. 

Considerations/questions:

- The test suite might leverage the existing Gingko and Gomega frameworks or anything else. Steps on writing an OSD addon operator test suite is provided here.

- An example operator test-suit-harness can be found here.