We need to be able to test refreshed or changed cockpit/tasks containers automatically and systematically, instead of the current "deploy it and watch out for troubles" cowboying. This cannot run on workflows or petty infrastructure, but needs to run on our main work horses (CentOS CI and e2e) due to /dev/kvm, access to image cache, etc.

Rough idea:

• Push new tasks containers to quay.io with a :staging tag
• Deploy a webhook and two tasks containers on CentOS CI from that tag. webhook should be accessible through a "webhook-staging" route. Deploy two staging tasks containers on e2e. This should all be in an Ansible Playbook.
• Set up cockpituous' webhook to point to webhook-staging, either permanently or as part of the playbook.
• Define a set of standard test cases, e. g. {cockpit, composer, starter-kit}

  x

  {public, rhel}

  x

  {chromium, firefox}

  and add that to tests-scan or tests-trigger (against a cockpituous PR).. However, this should not run on every cockpituous PR, just when we want it (CLI command is fine); possibly, image:tasks or container:tasks?

• A container refresh would then always need a PR to receive the test results. That's fine IMHO, and we can automate it similar to VM image refreshes.