We need a consistent way to allow pipeline owners to add test jobs to their repos for testing resolution and build of packages before merging changes in their requirements file. This should not be the default, but should be an option.

We need to add a new job template, enable it only when a flag is set when the jobs API file is included, etc.