Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4325

Setting custom options on the application tenant removes user-alertmanager configuration

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Hide
      Before this update, an issue in the Loki Operator caused the alert-manager configuration for the "application" tenant to disappear if the LokiStack was configured with additional options for that tenant. With this update, the generated Loki configuration now contains both the custom and the auto-generated configuration.
      Show
      Before this update, an issue in the Loki Operator caused the alert-manager configuration for the "application" tenant to disappear if the LokiStack was configured with additional options for that tenant. With this update, the generated Loki configuration now contains both the custom and the auto-generated configuration.
    • Bug Fix
    • Log Storage - Sprint 239, Log Storage - Sprint 240

      Description of problem:

      When a LokiStack is configured in openshift-logging tenant mode with a ruler and there is an alertmanager for user workloads present in the cluster, the Loki Operator automatically configures the application tenant to use that alertmanager by adding configuration to the runtime-config.yaml in the ConfigMap used by Loki.

      Example LokiStack:

      apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
  name: lokistack-dev
  namespace: openshift-logging
spec:
  size: 1x.extra-small
  storage:
    schemas:
    - version: v12
      effectiveDate: 2022-06-01
    secret:
      name: test
      type: s3
  storageClassName: gp3-csi
  tenants:
    mode: openshift-logging
  rules:
    enabled: true
    selector:
      matchLabels:
        openshift.io/cluster-monitoring: "true"
    namespaceSelector:
      matchLabels:
        openshift.io/cluster-monitoring: "true"

      Generated configuration:

      ---
overrides:
  application:
    ruler_alertmanager_config:
      alertmanager_url: https://_web._tcp.alertmanager-operated.openshift-user-workload-monitoring.svc
      enable_alertmanager_v2: true
      enable_alertmanager_discovery: true
      alertmanager_refresh_interval: 1m
      alertmanager_client:
        tls_ca_path: /var/run/ca/alertmanager/service-ca.crt
        tls_server_name: alertmanager-user-workload.openshift-user-workload-monitoring.svc.cluster.local
        type: Bearer
        credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token

      When the LokiStack is modified to contain overrides for the application tenant, for example a custom query-timeout, the alertmanager configuration vanishes from the configuration file and is replaced by the timeout:

      ---
overrides:
  application:
    query_timeout: 5m

      The configuration should actually contain both configuration options, the timeout and the alertmanager configuration, unless a custom alertmanager configuration is added by the user.

      Version-Release number of selected component (if applicable):

      How reproducible:

      Steps to Reproduce:

      1. Create LokiStack on OpenShift with user-workload monitoring enabled on the cluster and a custom queryTimeout for the application tenant
      2. Observe the contents of the generated ConfigMap

      Actual results:

      Generated ConfigMap contains only part of the configuration.

      Expected results:

      Generated ConfigMap contains both the autogenerated alertmanager configuration and the custom query-timeout.

      Additional info:

      LokiStack with custom queryTimeout:

      apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
  name: lokistack-dev
  namespace: openshift-logging
spec:
  limits:
    tenants:
      application:
        queries:
          queryTimeout: 5m
  managementState: Managed
  rules:
    enabled: true
    namespaceSelector:
      matchLabels:
        openshift.io/cluster-monitoring: "true"
    selector:
      matchLabels:
        openshift.io/cluster-monitoring: "true"
  size: 1x.extra-small
  storage:
    schemas:
    - effectiveDate: "2022-06-01"
      version: v12
    secret:
      name: test
      type: s3
  storageClassName: gp3-csi
  tenants:
    mode: openshift-logging

              rojacob@redhat.com Robert Jacob
              rojacob@redhat.com Robert Jacob
              Anping Li Anping Li
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: