Not to be confused with "flakes" in sippy, where a test is run multiple times and eventually passes.

We have many tests that will fail once, and then immediately succeed thereafter. Sippy has the data if we can figure out how to query it.

Rough proposal:

• query all runs of a test in the last X days, grouped by job.
• look for occurrences where the test fails exactly once, then immediately passes again within that particular job.
• repeat for every group by job that was returned.
• calculate a score for how often the test fails and then immediately heals, store it in the db.

This will take a fair bit of time likely going test by test.

This would bring some benefits:

• good indicator for the tests we need to fix or eliminate
• opens the door to categorize job failures, find those that might be safe to ignore, vs those that are truly weird and interesting and in need of attention