IBM regressive estimator: https://github.com/foundation-model-stack/fm-training-estimator

From this code and IBM presentation, we got curious about other methods to estimate resources for different cases (mixed_precision, which estimator...). 

Main source: https://blog.eleuther.ai/transformer-math/

The main issue is how to quantify CPU offloading.