Uploaded image for project: 'OpenShift GitOps'
  1. OpenShift GitOps
  2. GITOPS-5970

ArgoCD Notification Controller and having more than 4 contexts defined in the NotificationCOnfiguration, this causes the corresponding configmap to be rewritten multiple times before settling.

XMLWordPrintable

    • 5
    • False
    • None
    • False
    • Hide
      Before this update, notification Controller reconciled indefinitely if context field contains multiple entries and creates huge amount of logs. With this update, controller reconciliation is fixed and will stop the repeated log entries if multiple context is used.
      Show
      Before this update, notification Controller reconciled indefinitely if context field contains multiple entries and creates huge amount of logs. With this update, controller reconciliation is fixed and will stop the repeated log entries if multiple context is used.
    • GitOps Crimson - Sprint 3267, GitOps Crimson - Sprint 3268, GitOps Crimson - Sprint 3269

      *#Issue: *

      When using the ArgoCD Notification Controller and having more than 4 contexts defined in the NotificationConfiguration, this causes the corresponding configmap to be rewritten multiple times before settling.

      This issue causes the API to be hit with unnecessary request and causes a high amount of logs that could eventually fill up the cluster logging storage.
      [See the shared "argocd-notifications-controller" logs].
      If multiple projects on the same cluster use this feature with lots of context entries this could cause stability issues due to high API requests. This in turn could lead to outages of production systems.

      #Environment:

      RHOCP: 4.14.38
      Red Hat OpenShift GitOps : 1.14.0

      #Additional Details:

        • How to reproduce:*

      Scenario 1 (all works ok):

      1. Create a new namespace with ArgoCD with Notification Controller enabled
      2. Edit the NotificationConfiguration and add one context. ( See shared notificationsconfiguration-working for reference).
      3. Observe argocd-notifications-controller logs -> Single entry that cache gets invalidated due to configmap change.
      4. Observe argocd-notifications-cm configmap -> Configmap gets updated once.

      Scenario 2 (faulty):

      1. Edit the NotificationConfiguration and add more context entries. ( See shared notificationsconfiguration-faulty for reference).
      2. Observe argocd-notifications-controller logs -> Fast stream of log lines showing that the cache gets invalidated over and over due to configmap change.
      3. Observe argocd-notifications-cm configmap -> Configmap gets updated hundred or thousands of times.
      -> In this Scenario after a random amount of time the configmap will stop updating, but this is non deterministic and certainly a faulty behavior.

              rh-ee-ansingh Anand Singh
              rhn-support-dkarde Dipak Karde
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: