Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-7887

Can't forward logs to cloudwatch/azureMonitor when `networkPolicy.ruleSet` is `RestrictIngressEgress`.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • None
    • Logging 6.4.0
    • Log Collection
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • NEW
    • Bug Fix
    • Critical

      Description of problem:

      Collector pods can't forward logs to cloudwatch/azureMonitor when `networkPolicy.ruleSet` is `RestrictIngressEgress` and raise below errors:

      2025-10-15T02:08:12.765835Z  WARN sink{component_kind="sink" component_id=output_cloudwatch component_type=aws_cloudwatch_logs}: vector::sinks::util::retries: Retrying after error. error=CloudwatchError::Put: dispatch failure internal_log_rate_limit=true
      2025-10-15T02:08:13.281222Z  WARN sink{component_kind="sink" component_id=output_cloudwatch component_type=aws_cloudwatch_logs}: vector::sinks::util::retries: Internal log [Retrying after error.] is being suppressed to avoid flooding.
      2025-10-15T02:08:22.904893Z  WARN sink{component_kind="sink" component_id=output_cloudwatch component_type=aws_cloudwatch_logs}: vector::sinks::util::retries: Internal log [Retrying after error.] has been suppressed 2 times.
      2025-10-15T02:08:22.904909Z  WARN sink{component_kind="sink" component_id=output_cloudwatch component_type=aws_cloudwatch_logs}: vector::sinks::util::retries: Retrying after error. error=CloudwatchError::DescribeLogStreams: dispatch failure internal_log_rate_limit=true
      2025-10-15T02:08:29.786698Z  WARN sink{component_kind="sink" component_id=output_cloudwatch component_type=aws_cloudwatch_logs}: vector::sinks::util::retries: Internal log [Retrying after error.] is being suppressed to avoid flooding. 

       

      2025-10-15T09:17:55.578236Z  WARN sink{component_kind="sink" component_id=output_azure_infra component_type=azure_monitor_logs}: vector::internal_events::http_client: Internal log [HTTP error.] has been suppressed 1 times.
      2025-10-15T09:17:55.578269Z  WARN sink{component_kind="sink" component_id=output_azure_infra component_type=azure_monitor_logs}: vector::internal_events::http_client: HTTP error. error=error trying to connect: dns error: failed to lookup address information: Name or service not known error_type="request_failed" stage="processing" internal_log_rate_limit=true
      2025-10-15T09:17:55.578338Z  WARN sink{component_kind="sink" component_id=output_azure_infra component_type=azure_monitor_logs}: vector::sinks::util::retries: Internal log [Retrying after error.] has been suppressed 1 times.
      2025-10-15T09:17:55.578346Z  WARN sink{component_kind="sink" component_id=output_azure_infra component_type=azure_monitor_logs}: vector::sinks::util::retries: Retrying after error. error=Failed to make HTTP(S) request: error trying to connect: dns error: failed to lookup address information: Name or service not known internal_log_rate_limit=true
      2025-10-15T09:18:01.714169Z  WARN sink{component_kind="sink" component_id=output_azure_infra component_type=azure_monitor_logs}: vector::internal_events::http_client: Internal log [HTTP error.] is being suppressed to avoid flooding.
      2025-10-15T09:18:01.714238Z  WARN sink{component_kind="sink" component_id=output_azure_infra component_type=azure_monitor_logs}: vector::sinks::util::retries: Internal log [Retrying after error.] is being suppressed to avoid flooding. 

      Version-Release number of selected component (if applicable):

      cluster-logging.v6.4.0

      How reproducible:

      Always 

      Steps to Reproduce:

      1. Create CLF to forward logs to cloudwatch with below yaml:

      apiVersion: observability.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        name: clf-74926
      spec:
        collector:
          networkPolicy:
            ruleSet: RestrictIngressEgress
        managementState: Managed
        outputs:
        - cloudwatch:
            authentication:
              awsAccessKey:
                keyId:
                  key: aws_access_key_id
                  secretName: logging-74926-pqb41t8d
                keySecret:
                  key: aws_secret_access_key
                  secretName: logging-74926-pqb41t8d
              type: awsAccessKey
            groupName: logging-74926-qitang-pkf7h.{.log_type||"none-typed-logs"}
            region: us-east-2
            url: https://logs.us-east-2.amazonaws.com
          name: cloudwatch
          type: cloudwatch
        pipelines:
        - inputRefs:
          - infrastructure
          - audit
          - application
          name: to-cloudwatch
          outputRefs:
          - cloudwatch
        serviceAccount:
          name: cloudwatch-sn51mq0v 

      2. Check logs in cloudwatch
      3. Check collector pods' log

       

      Actual results:

      No logs found in cloudwatch/azureMonitor, collector pods raise some errors.

      Expected results:

      Logs can be forward log cloudwatch/azureMonitor without any errors.

      Additional info:

      No issue when ruleSet is AllowAllIngressEgress.

              Unassigned Unassigned
              qitang@redhat.com Qiaoling Tang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: