When changing numvfs, there are cases that the change is not reflexed in a mlx nic. Consequently, the node will get into a boot loop. At a minimum, we need to detect that changing the setting failed, and stop boot-looping with an error message.

Cold reboot also fixes the issue. Need to investigate if it's possible to properly fix this state too w/o cold reboot.