Uploaded image for project: 'OpenShift GitOps'
  1. OpenShift GitOps
  2. GITOPS-2376

ExternalSecret is OutOfSync most of the time

XMLWordPrintable

    • 5
    • False
    • None
    • False
    • GITOPS Sprint 236

       Syncing the following resource:

       

      apiVersion: external-secrets.io/v1alpha1
      kind: ExternalSecret
      metadata:
        name: fluent-bit-audit
        namespace: fluent-bit
        labels:
          app.kubernetes.io/name: fluent-bit-audit
      spec:
        refreshInterval: 1h
        secretStoreRef:
          name: ocpapp-cluster-secret-store
          kind: ClusterSecretStore
        target:
          name: fluent-bit-audit
          creationPolicy: Owner
          template:
            metadata:
              labels:
                app.kubernetes.io/name: fluent-bit-audit
            type: Opaque
            data:
              fluent-bit.conf: |-
                [SERVICE]
                    Daemon                       Off
                    Flush                        5
                    Log_Level                    info
                    Parsers_File                 parsers.conf
                    Parsers_File                 custom_parsers.conf
                    HTTP_Server                  On
                    HTTP_Listen                  0.0.0.0
                    HTTP_Port                    2020
                    Health_Check                 On          [INPUT]
                    Name              forward
                    Listen            0.0.0.0
                    Port              24224
                    Buffer_Chunk_Size 1M
                    Buffer_Max_Size   30M          [OUTPUT]
                    Name        azure
                    Match       *
                    Customer_ID {{ .LAWS_WSID | toString }}
                    Shared_Key  {{ .LAWS_KEY | toString }}
                    Log_Type    ocp_audit
                    Retry_Limit 3          [OUTPUT]
                    name          splunk
                    match         *
                    host          {{ .SPLUNK_HOST | toString }}
                    port          8088
                    event_index   apigateway_test_openshiftaudit
                    splunk_token  {{ .SPLUNK_KEY | toString }}
                    tls           on
                    tls.verify    off
                    Retry_Limit 3          [OUTPUT]
                    name                  azure_blob
                    match                 linux-audit*
                    account_name          {{ .LOG_RETN_BLOB_NAME | toString }}
                    shared_key            {{ .LOG_RETN_BLOB_KEY | toString }}
                    blob_type             blockblob
                    path                  ocp-audit/linux-audit
                    container_name        ocp-audit
                    tls                   on
                    Retry_Limit           3          [OUTPUT]
                    name                  azure_blob
                    match                 k8s-audit*
                    account_name          {{ .LOG_RETN_BLOB_NAME | toString }}
                    shared_key            {{ .LOG_RETN_BLOB_KEY | toString }}
                    blob_type             blockblob
                    path                  ocp-audit/k8s-audit
                    container_name        ocp-audit
                    tls                   on
                    Retry_Limit           3          [OUTPUT]
                    name                  azure_blob
                    match                 openshift-audit*
                    account_name          {{ .LOG_RETN_BLOB_NAME | toString }}
                    shared_key            {{ .LOG_RETN_BLOB_KEY | toString }}
                    blob_type             blockblob
                    path                  ocp-audit/openshift-audit
                    container_name        ocp-audit
                    tls                   on
                    Retry_Limit           3          [OUTPUT]
                    name                  azure_blob
                    match                 kubernetes*
                    account_name          {{ .LOG_RETN_BLOB_NAME | toString }}
                    shared_key            {{ .LOG_RETN_BLOB_KEY | toString }}
                    blob_type             blockblob
                    path                  ocp-audit/application-audit
                    container_name        ocp-audit
                    tls                   on
                    Retry_Limit           3
        data:
        - secretKey: LAWS_WSID
          remoteRef:
            key: log-analytics-workspace-id-dc1
        - secretKey: LAWS_KEY
          remoteRef:
            key: log-analytics-workspace-key-dc1
        - secretKey: SPLUNK_HOST
          remoteRef:
            key: splunk-host-dc1
        - secretKey: SPLUNK_KEY
          remoteRef:
            key: splunk-key-dc1
        - secretKey: LOG_RETN_BLOB_NAME
          remoteRef:
            key: log-retn-blob-name-dc1
        - secretKey: LOG_RETN_BLOB_KEY
          remoteRef:
            key: log-retn-blob-key-dc1 

       

      Utilizing External Secrets in strategy to access secrets from Azure Key Vault.
      In ExternalSecrets, there is a resource `.spec.refreshInterval`. This value is default set at 1h. Our CRDs Also set this at 1h. The issue is that it is being parsed to `1h0m0s`. As such this is causing ArgoCD (OpenShift GitOps) to flag it as 'out of sync'.

      Tried the workaround mentioned in the ticket GITOPS-2211 but it didn't work.

      It is noted that External Secrets does have its own webhook that notices the same issue when deploying a CRD which is `apiVersion: external-secrets.io/v1alpha1`. As the external secrets operator converts to `apiVersion: external-secrets.io/v1beta1`.

      However instead of ArgoCD noticing the live manifest of `apiVersion: external-secrets.io/v1beta1` it ignores the managedFields update which progresses from `apiVersion: external-secrets.io/v1alpha1` to `apiVersion: external-secrets.io/v1beta1`.

      Even if the live manifest is physically updated to the repository format ArgoCD does not observe the change.

      Business Impact:
      DevOps support team is being distracted by out of sync flags for applications that are in-sync.

              Unassigned Unassigned
              rhn-support-sburhade Satyam Burhade
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: