Story (Required)

As a maintainer of openshift-gitops, I would like to run kuttl tests and argocd e2e tests automatically before a release goes out (whether it is automated e.g. BOTA or y-stream/z-stream). 

Our plumbing test (aka acceptance test) are the "external test" following the third approach as described in here.

In this story: 2796, we add the necessary artifacts to the Plumbing test infrastructure to run acceptance tests (the "external tests" in the context of CVP) after being triggered by a UMB message in the topic VirtualTopic.eng.ci.redhat-container-image.index.built.

We need to send the results of that testing to ResultsDB also via UMB message.

Background (Required)

See parent epic

Approach (Required)

Creating a python script to send the message properly formatted.

More information about this method here.

Specific details about the trigger setup can be found here: https://gitlab.cee.redhat.com/cvp/pipeline/blob/master/samples/umb-interactions/Job_DSL.groovy#L16

Example of message content

messageContent:
{
    "timestamp": "2019-09-18T12:20:35.497589Z",
    "version": "0.1.0",
    "generated_at": "2019-09-18T12:20:35.497589Z",
    "ci": {
        "name": "Container Verification Pipeline",
        "team": "CVP Development Team",
        "doc": "https://docs.engineering.redhat.com/display/CVP/Container+Verification+Pipeline+E2E+Documentation",
        "url": "https://jenkins-cvp-5d0cbc05bed8bd5717e3e5b4.apps.ocp-c1.prod.psi.redhat.com/",
        "email": "cvp-ops@redhat.com"
    },
    "run": {
        "url": "https://jenkins-cvp-5d0cbc05bed8bd5717e3e5b4.apps.ocp-c1.prod.psi.redhat.com/job/cvp-product-test/15/",
        "log": "https://jenkins-cvp-5d0cbc05bed8bd5717e3e5b4.apps.ocp-c1.prod.psi.redhat.com/job/cvp-product-test/15/console"
    },
    "artifact": {
        "type": "cvp",
        "id": "970279",
        "component": "cvp-openshiftserverless",
        "issuer": "freshmaker",
        "brew_build_target": "openshift-serverless-1-rhel-8-containers-candidate",
        "brew_build_tag": "openshift-serverless-1-rhel-8-container-candidate",
        "nvr": "openshift-serverless-1-tech-preview-serving-activator-rhel8-container-0.7.1-4.1568806559",
        "full_name": "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift-serverless-1-tech-preview/serving-activator-rhel8:0.7.1-4.1568806559",
        "registry_url": "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888",
        "namespace": "openshift-serverless-1-tech-preview",
        "name": "serving-activator-rhel8",
        "image_tag": "0.7.1-4.1568806559",
        "advisory_id": "45776",
        "scratch": "false"
    },
    "pipeline": {
        "name": "cvp-product-test",
        "id": "cbc62d7b-2a0f-440b-a5d4-7ac443f870e3",
        "status": "running",
        "build": "15"
    }
} 

You can find all the related files in cvp/pipeline git repository:
https://gitlab.cee.redhat.com/cvp/pipeline/-/tree/master/samples/umb-interactions

Acceptance Criteria (Mandatory)

• Results of the test triggered by UMB topic VirtualTopic.eng.ci.redhat-container-image.index.built are sent back to UMB to update ResultsDB, to the respective topics:
  • VirtualTopic.eng.ci.gitops-external-tests.pipeline.validation.test.complete
  • VirtualTopic.eng.ci.gitops-external-tests.pipeline.validation.test.error
• Python script has some unit tests to verify its basic functionality

INVEST Checklist

Dependencies identified

Blockers noted and expected delivery timelines set

Design is implementable

Acceptance criteria agreed upon

Story estimated

Done Checklist

• Code is completed, reviewed, documented and checked in
• Unit and integration test automation have been delivered and running cleanly in continuous integration/staging/canary environment
• Continuous Delivery pipeline(s) is able to proceed with new code included
• Customer facing documentation, API docs etc. are produced/updated, reviewed and published
• Acceptance criteria are met