Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2418

EO supported time units don't match the units specified in CRDs.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • Logging 5.5.0
    • Logging 5.4.0
    • Log Storage
    • None
    • Logging (LogExp) - Sprint 217, Logging (LogExp) - Sprint 218

      Description of problem:

      In crd/elasticsearches.logging.openshift.io and crd/clusterloggings.logging.openshift.io, the supported time units for logStore.retentionPolicy.application.maxAge are yMwdhHms, however when setting maxAge to `1M`, the cronjob can't be created, EO keeps reporting following error:

      {"_ts":"2022-03-28T08:23:44.516794667Z","_level":"0","_component":"elasticsearch-operator","_message":"could not reconcile indexmanagement cronjob","_error":{"msg":"conversion to millis for time unit is unsupported","timeunit":"M"},"mapping":"app","policy":"app-policy"}
      {"_ts":"2022-03-28T08:23:44.516877553Z","_level":"0","_component":"elasticsearch-operator_controller_elasticsearch-controller","_message":"Reconciler error","_error":{"msg":"conversion to millis for time unit is unsupported","timeunit":"M"},"name":"elasticsearch","namespace":"openshift-logging"} 

      In crd:

                        retentionPolicy:
                          description: Retention policy defines the maximum age for an index
                            after which it should be deleted
                          nullable: true
                          properties:
                            application:
                              nullable: true
                              properties:
                                maxAge:
                                  description: TimeUnit is a time unit like h,m,d
                                  pattern: ^([0-9]+)([yMwdhHms]{0,1})$
                                  type: string
                                namespaceSpec:
                                  description: The per namespace specification to delete
                                    documents older than a given minimum age
                                  items:
                                    properties:
                                      minAge:
                                        description: Delete the records matching the namespaces
                                          which are older than this MinAge (e.g. 1d)
                                        pattern: ^([0-9]+)([yMwdhHms]{0,1})$
                                        type: string
                                      namespace:
                                        description: Target Namespace to delete logs older
                                          than MinAge (defaults to 7d) Can be one namespace
                                          name or a prefix (e.g., "openshift-" covers all
                                          namespaces with this prefix)
                                        type: string
                                    required:
                                    - namespace
                                    type: object
                                  type: array
                                pruneNamespacesInterval:
                                  description: How often to run a new prune-namespaces job
                                  pattern: ^([0-9]+)([yMwdhHms]{0,1})$
                                  type: string
                              type: object 

      Version-Release number of selected component (if applicable):

      cluster-logging.5.4.0-109 

      elasticsearch-operator.5.4.0-123 

      How reproducible:

      Always

      Steps to Reproduce:
      1. subscribe CLO and EO
      2. create clusterlogging with:

      apiVersion: "logging.openshift.io/v1"
      kind: "ClusterLogging"
      metadata:
        name: "instance"
        namespace: "openshift-logging"
      spec:
        managementState: "Managed"
        logStore:
          type: "elasticsearch"
          retentionPolicy: 
            application:
              maxAge: 6M
              namespaceSpec:
              - minAge: 1M
                namespace: project-1
              - minAge: 2M
                namespace: project-2
              pruneNamespacesInterval: 30m
            infra:
              maxAge: 3M
            audit:
              maxAge: 10d
          elasticsearch:
            nodeCount: 1
            redundancyPolicy: "ZeroRedundancy"
            resources:
              requests:
                cpu: 1
                memory: "2Gi"
            storage:
              storageClassName: "standard"
              size: "20Gi"
        visualization:
          type: "kibana"
          kibana:
            replicas: 1
        collection:
          logs:
            type: "fluentd"
            fluentd: {}

      3. check cronjobs and logs in EO

      Actual results:

      In elasticsearch CR, the policies are generated:

            policies:
            - name: app-policy
              phases:
                delete:
                  minAge: 6M
                  namespaceSpec:
                  - minAge: 1M
                    namespace: project-1
                  - minAge: 2M
                    namespace: project-2
                  pruneNamespacesInterval: 30m
                hot:
                  actions:
                    rollover:
                      maxAge: 9d
              pollInterval: 15m
            - name: infra-policy
              phases:
                delete:
                  minAge: 30d
                  pruneNamespacesInterval: 30m
                hot:
                  actions:
                    rollover:
                      maxAge: 1d
              pollInterval: 15m
            - name: audit-policy
              phases:
                delete:
                  minAge: 10d
                  pruneNamespacesInterval: 30m
                hot:
                  actions:
                    rollover:
                      maxAge: 12h
              pollInterval: 15m 

      however, no cronjobs:

      $ oc get cj
      No resources found in openshift-logging namespace. 

      Expected results:

      Additional info: 

            btaani@redhat.com Bayan Taani
            qitang@redhat.com Qiaoling Tang
            Qiaoling Tang Qiaoling Tang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: