Description of problem:

The same application requires different memory size in different namespaces.

Our application team tried to deploy an image to two different namespaces. However, we found that the deployed pods (despite coming from the same image) have different memory consumption across different namespaces; one pod spends +- 25 Mb memory in namespace "A", while the other spends 300 Mb memory in namespace "B".

We tried to redeploy the image several times; we tried pulling the image from namespace A to B and deploy it (and vice versa), we also tried to use different worker nodes and made sure that the pods - despite being deployed in different projects - are deployed in the same worker node. We found that whatever it is that we do, the pods in namespace B always requires 10x the memory needed for namespace A - no matter which worker hosts the container.

Several things to note:
1. Through deploying the app to a third project, we found that the app consumes +- 25 Mb, which means that the namespace exhibiting abnormal behavior is likely namespace B.
2. Namespace A only contains 2 - 3 deployments, meanwhile namespace B has > 700 pods with average memory consumption of 300 - 500 Mb. We do suspect that there is something going on with namespace B, however deploying a sample app to both namespaces shows no difference in memory consumption.
3. The app memory consumption of 300 Mb happens if there is no limit being set. We found that memory consumption in namespace B is decreasing through setting a memory limit of 300 Mb (memory decreases from 300 Mb to 45 Mb), however the container will crash if we set the limit below 300 Mb. We do not find this behavior in namespace A or the third project; the app runs smoothly even with a very low memory limit being set.

Version-Release number of selected component (if applicable):

How reproducible:

NA

Steps to Reproduce:

1.
2.
3.

Actual results:

Expected results:

Additional info: