Uploaded image for project: 'Observability and Data Analysis Program'
  1. Observability and Data Analysis Program
  2. OBSDA-1206

Reducing Log Loss during node scaledown by ensuring Vector terminates last

XMLWordPrintable

    • Icon: Feature Feature
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • Log Collection
    • False
    • Hide

      None

      Show
      None
    • False
    • Not Selected
    • 0

      Proposed title of this feature request

      Reducing Log Loss During Node Scale-Down by Ensuring Vector Terminates Last

      What is the nature and description of the request?

      Problem Statement:
      During node shutdown, the Collector pod terminates before other workload pods leading to log loss. Ideally, the Collector pod should be the last to terminate, ensuring that workload logs are captured and forwarded before shutdown.

      Reproduction steps done for validation of Log Loss :

      1> Deployed a log-generator application that continuously writes logs to both Persistent Volume and STDOUT (10 logs/sec).

      2> Initiated a shutdown of the node hosting this log-generator pod

      Compared the logs:

      • PV recorded up to line 95999
      • Loki recorded only up to line 93627
      • This confirms a loss of 2372 log lines during shutdown.

      Why does the customer need this? (List the business requirements)

      To avoid logloss during the scaledown event

      List any affected packages or components.

      Vector, Node, Crio

              Unassigned Unassigned
              rhn-support-anisal Apurva Nisal
              Votes:
              11 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: