Currently the scheduler uses a simple polling mechanism to determine when an export is completed. This will likely work fine for export jobs that run/finish quickly. However, it sounds like Cost exports might take hours to complete.

We need to come up with an approach that allows for handling jobs that finish quickly and jobs that take hours to complete.

We need to also consider that the scheduler could be restarted during the polling time.

Also, if a scheduled job is deleted, then we need to stop/cancel the polling.

Also, consider adding event completion notifications to the export service. Maybe that simplifies this and export events are probably a nice thing to have in general.