We have three data stores for test data (gce-devel bigquery, ci-data bigquery, sippy), each of which has different rules for what jobs get imported, and different code determining what variants a job should be categorized by.

Why is this job not in a place we expect, and why did it get miscategorized is a frequent source of bugs and issues.

Goal of this card is to come up with a central location where the job variants are defined, and what should be imported where. Variants change, we calculate differently, add new, update logic, thus we do not want to embed this data with actual job run data as we may wish to recategorize a job later on. It also does not fit nicely into an uploader routine.

Because sippy pulls data from bigquery, and the other two data sources are in bigquery, bigquery is the most logical place for this. Sippy's categorization is most robust so we should likely try to re-use it's logic.

The data store should be synced on some interval by examining what jobs exist, and categorizing their variants. This could potentially source from the release repo, or by looking for the unique jobs list we see in the test run data in gce-devel.

Ensure we can join across bigquery/gcp projects so the one table can be used for both.

Update all three data stores to use this for "what jobs should I import" and "what should their variants be".