As seen in CRW-2602, crwctl can hit a wall building when it includes older tags from the crw-images repo, created as part of the process of publishing asset files.

So... to work around this short term, we can run crwctl builds on the dynamic nodes only.

But a more complete solution would be to use:

• a unique yarn cache for each build on the permanent nodes, and
• delete the per-build cache when done.

This would also free up disk space as ~/.cache/yarn/ is 11G on x86-64-rhel8-p2 and 6.8G on x86-64-rhel8-p1

TODO:

• Add something to the crontab/reboot scripts - https://gist.github.com/nickboldt/bb366f9acd2995e18ed0f836f1cb9307
• remount the container volume from ppc006 so it's usable space