We have a testing gate in check-payload that executes unit tests, which is great. It would be nice to supplement this with a test job that executes check-payload on a payload, pre-merge.

We have some of this already defined and configured in openshift/releases for testing the payloads. We just need to port that testing configuration to the check-payload repository so that we can run the same test pre-merge.

This should help us avoid regressions in the tool that have adverse side-effects on the overall OpenShift check-payload testing.