System application is eating memory overtime, it needs to be restarted from time to time on SaaS.

It looks that only processes involved in rendering templates are having this issue.
Also On-Premises installation did not report any issues about it, so it has to do with SaaS particular environment

Facts:

• Sidekiq processes sending emails have memory leak
• Unicorn processes for web rendering have memory leak
• Unicorn processes for API endpoints DO NOT have memory leak
• Only with a high number of tenants there is memory leak

Added 5 December, 2019 - See also this comment in THREESCALE-3792

Dev notes

After looking at the code, there is one particular part that does caching based on tenant ID

https://github.com/3scale/porta/blob/109600f4aefdd174ddd0179c06f65c94850421cd/lib/developer_portal/lib/liquid/template/resolver.rb#L15-L18

Rails is already doing some caching underneath but here we also do another caching per tenant.

Steps to reproduce

• Create thousands of tenants
• Query developer portal pages of each of those tenants
• See memory increasing. See this comment

Fixing

Implement LRU caching on these objects or just removing the cache.
Negative impact is probably the performance for rendering templates for developer portal
Add some counter-measure to fix performance rendering (partial rendered pages?)