Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-7013

Increase MAX_EVENT_SIZE from 256KB to 1MB for CloudWatch output

XMLWordPrintable

    • Future Sustainability
    • 3
    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • VERIFIED
    • With this update the Cloudwatch output now supports log events up to 1 MB in size, a 4x increase from the previous 256 KB limit. This matches the recent updates published by AWS.
    • Bug Fix
    • Log Collection - Sprint 269, Log Collection - Sprint 270
    • Moderate

      Description of problem:

      In the Vector collector pods is observed errors as reached the MAX_EVENT_SIZE: 256KB when log forwarding to cloudwatch:

      2025-04-11T06:38:37.368665Z ERROR sink{component_kind="sink" component_id=output_cloudwatch component_type=aws_cloudwatch_logs}: vector::internal_events::aws_cloudwatch_logs: Encoded event is too long. size=354390 max_size=262094 error_code="message_too_long" error_type="encoder_failed" stage="processing" internal_log_rate_limit=true 

      The error is coded in the "aws_cloudwatch_logs.rs" and it's set as a constant where the MAX_EVENT_SIZE is set to 256 * 1024 in this line

      A couple of days ago, Amazon CloudWatch Logs increased the maximum log event size to 1 MB as described in the next link. Then, it should be needed to adapt the MAX_EVENT_SIZE in Vector when log forwarding to CloudWatch to be 1 MB.

      Version-Release number of selected component (if applicable):

      All the Logging v5 versions and Logging 6.0.z, 6.1.z and 6.2.z

      How reproducible:

      Always

      Steps to Reproduce:

      1. Configure the log collector to log forward to Cloudwatch
      2. Generate an event in one application where the event size is bigger than 256 KB
      3. Review the collector logs

      Actual results:

      The event fails to be sent with the error:

      2025-04-11T06:38:37.368665Z ERROR sink{component_kind="sink" component_id=output_cloudwatch component_type=aws_cloudwatch_logs}: vector::internal_events::aws_cloudwatch_logs: Encoded event is too long. size=354390 max_size=262094 error_code="message_too_long" error_type="encoder_failed" stage="processing" internal_log_rate_limit=true  

      Expected results:

      The event is log forwarded always that not exceeding the new Amazon CloudWatch log event size of 1 MB.

      Additional info:

              cahartma@redhat.com Casey Hartman
              rhn-support-ocasalsa Oscar Casal Sanchez
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: