first determine if the job is worth keeping around or just disable entirely.

if we will keep it, and want it required, file bugs and resolve them until it
passes at a higher rate (> 80-90%). Make the job optional in the meantime

spending cycles on /retest because we know a job will eventually pass is
gives a false sense of a PRs health. The jobs need to be counted on and
taken serious when they fail.