Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4491

Change of default settings to reduce 429 errors on install

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Undefined Undefined
    • None
    • Logging 5.7.5
    • Log Storage
    • False
    • None
    • False
    • NEW
    • NEW

      Description of problem: After testing on the CU side, the settings found here: https://github.com/grafana/loki/issues/4204, specifically;

      retention_period: 72h
      enforce_metric_name: false
      reject_old_samples: true
      reject_old_samples_max_age: 168h
      max_cache_freshness_per_query: 10m
      split_queries_by_interval: 15m

      1. for big logs tune
        per_stream_rate_limit: 512M
        per_stream_rate_limit_burst: 1024M
        cardinality_limit: 200000
        ingestion_burst_size_mb: 1000
        ingestion_rate_mb: 10000
        max_entries_limit_per_query: 1000000
        max_label_value_length: 20480
        max_label_name_length: 10240
        max_label_names_per_series: 300

      Helped in completely eliminating the 429 errors that were filling the logs since install. The CU would like to make a case to instill these as the 'default' settings.

      Version-Release number of selected component (if applicable): 
      
      Loki Operator 5.7
      
      

      How reproducible: Easy

      Steps to Reproduce:
      1. Setup Openshift Lab
      2. Install Loki Operator 
      3. Using default Loki Config, setup log fowarding and review operator logs
      

      Actual results: Logging works, but 429 errors are frequent

      Expected results: Very limited/completely absent of 429 errors in logs on base install
      
      

      Additional info: Related SF Case 03565373

      
      

            Unassigned Unassigned
            dacarpen@redhat.com Darren Carpenter
            Anping Li Anping Li
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: