Description of problem:

• After upgrading cluster to 4.12 version, nodes are getting scaled up automatically.
• Cu demonstrated by scaling down the machine count from 50 to 45. There are jobs very frequently running in namespace particular namespace and that causes to scaleup the machine count. Same can be found in the events section.
• Although machine scaleup happened, the pod which caused the scale up was still scheduled on other node and it got completed.
• There is no automatic scale down policy.

Workaround:

We have successfully created a temporary priority class that sets the value to -11, and had the application team update their job spec to leverage the new PC, which seems to have successfully stopped the scale-up events so far as expected.