-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
Logging 5.7.5
-
False
-
-
False
-
NEW
-
NEW
-
Bug Fix
-
-
When I do the upgrade to CLO 5.7.5, everything works fine for some minutes The errors appears but all the kafka's pods are up and functionals. When I do the rollback to CLO 5.6.10, I didn't need to do any actions on kafka pods. This is the third time I try to do this upgrade and the problem always occurs and is always the same.
2023-09-16T14:58:59.278883Z ERROR librdkafka: librdkafka: FAIL [thrd:kafka-ocp-kafka-0.kafka-ocp-kafka-brokers.kafka.svc:9092/0]: kafka-ocp-kafka-0.kafka-ocp-kafka-brokers.kafka.svc:9092/0: 1 request(s) timed out: disconnect (average rtt 2968.274ms) (after 351043ms in state UP)
2023-09-16T14:58:59.278929Z ERROR rdkafka::client: librdkafka: Global error: OperationTimedOut (Local: Timed out): kafka-ocp-kafka-0.kafka-ocp-kafka-brokers.kafka.svc:9092/0: 1 request(s) timed out: disconnect (average rtt 2968.274ms) (after 351043ms in state UP)
2023-09-16T14:58:59.282453Z ERROR sink\{component_kind="sink" component_id=kafka_app component_type=kafka component_name=kafka_app}: vector_common::internal_event::service: Service call failed. No retries or retries exhausted. error=Some(KafkaError (Message production error: MessageTimedOut (Local: Message timed out))) request_id=298501 error_type="request_failed" stage="sending" internal_log_rate_limit=true
2023-09-16T14:58:59.282505Z ERROR sink\{component_kind="sink" component_id=kafka_app component_type=kafka component_name=kafka_app}: vector_common::internal_event::component_events_dropped: Events dropped intentional=false count=1 reason="Service call failed. No retries or retries exhausted." internal_log_rate_limit=true
2023-09-16T14:58:59.282757Z ERROR sink\{component_kind="sink" component_id=kafka_app component_type=kafka component_name=kafka_app}: vector_common::internal_event::service: Internal log [Service call failed. No retries or retries exhausted.] is being rate limited.
2023-09-16T14:58:59.282828Z ERROR sink\{component_kind="sink" component_id=kafka_app component_type=kafka component_name=kafka_app}: vector_common::internal_event::component_events_dropped: Internal log [Events dropped] is being rate limited.