While looking at the bugzilla and the subsequent support cases we recognize a pattern where the WAL directory keeps growing and not being flushed even for days. The end result is that Prometheus ends up in an endless "OOM restart loop" because it never gets a chance to flush the WAL to disk.
We should investigate if we can implement some sort of init-container that prevents this symptom, by i.e. deleting the WAL in worst case.