Initial summary improvement Document for solving delete/insert summary task issues. This is a good thing to fix but is only looking at a small slice of the pie. May even be a bandaid style fix if we don't consider the relation to the rest of the pipeline.

We should spend a little time planning what we want next to improve the overall pipeline process and the ability to observe it. This may also include more robust book keeping of some kind and give us the ability to trigger any part of the pipeline and its subsequent tasks. For example if we need to reprocess data into Parquet, we know where that failed and triggering that specific step will also trigger any step after it like for example summary.

Lets design exactly how we think the new and improved pipeline should look to start! Once we have an overall plan agreed as a team we can talk through how each part could be broken up. This should give us a clear indication of where book keeping aspects could/should fit. After that we can look at breaking up the pipeline into clear implementation pieces.