-
Bug
-
Resolution: Unresolved
-
Major
-
1.14.0
-
False
-
None
-
False
-
-
*#Issue: *
When using the ArgoCD Notification Controller and having more than 4 contexts defined in the NotificationConfiguration, this causes the corresponding configmap to be rewritten multiple times before settling.
This issue causes the API to be hit with unnecessary request and causes a high amount of logs that could eventually fill up the cluster logging storage.
[See the shared "argocd-notifications-controller" logs].
If multiple projects on the same cluster use this feature with lots of context entries this could cause stability issues due to high API requests. This in turn could lead to outages of production systems.
#Environment:
RHOCP: 4.14.38
Red Hat OpenShift GitOps : 1.14.0
#Additional Details:
-
- How to reproduce:*
Scenario 1 (all works ok):
1. Create a new namespace with ArgoCD with Notification Controller enabled
2. Edit the NotificationConfiguration and add one context. ( See shared notificationsconfiguration-working for reference).
3. Observe argocd-notifications-controller logs -> Single entry that cache gets invalidated due to configmap change.
4. Observe argocd-notifications-cm configmap -> Configmap gets updated once.
Scenario 2 (faulty):
1. Edit the NotificationConfiguration and add more context entries. ( See shared notificationsconfiguration-faulty for reference).
2. Observe argocd-notifications-controller logs -> Fast stream of log lines showing that the cache gets invalidated over and over due to configmap change.
3. Observe argocd-notifications-cm configmap -> Configmap gets updated hundred or thousands of times.
-> In this Scenario after a random amount of time the configmap will stop updating, but this is non deterministic and certainly a faulty behavior.