Loading...

Details

Type: Bug
Resolution: Done
Priority: Critical
Fix Version/s: 1.6.1, 1.4.12, 1.5.6, 1.7.0
Affects Version/s: None
Component/s: Operator
Labels:
None

Story Points:
3
Blocked:
False
Blocked Reason:
None
Ready:
False
Release Note Text:
This fix removes the liveness probe in the application controller statefulset. This solves the problems of multiple restarts of application controllers in a large set of applications caused due to unresponsiveness of liveness probes.

Sprint:
GITOPS Sprint 222, GITOPS Sprint 223, GITOPS Sprint 224

SFDC Cases Counter:
SFDC Cases Links:

Description

Upstream has removed the liveness probe from the application controller's statefulset manifest with https://github.com/argoproj/argo-cd/pull/9557

Reasoning behind this is:

In cases when the controller is overloaded reconciling large queues restarting the Pod is more harmful than letting it running. In discussion with @alexmt we understand that liveness probe should be removed from the controller to prevent it from being restarted.

We should remove the liveness probe as well, the reasoning makes sense, and I have observed it multiple times now with a large set of apps that the application controller will be constantly restarted due to overload/unresponsiveness of the liveness probe.

This should also be backported to 1.4, 1.5 and 1.6 patch releases.