It seems we are incorrectly removing parquet files mid month because some AWS reports have invoice ID's within reports when we are not expecting them.

This is caused by multiple split report files from a cloud provider. one report file without invoice ids setting a processing range, and another report file with invoice ids setting the clear all parquet files flag to true. This results in us removing all files by mistake. Luckily there is no fall out immediately from this unless you manually run a summary task for the full month range.

Potential solution: Never set finalisation to run on current month since really there should be no final (full month reprocessing) mid month. This will prevent us inadvertently deleting parquet files when we shouldnt.