Uploaded image for project: 'Hybrid Cloud Console'
  1. Hybrid Cloud Console
  2. RHCLOUD-38736

Improve Kafka Producer Reliability: Add Logging, Try-Catch, and Retry Logic

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Hide

       

      • Errors during Kafka producer initialization are caught and logged.
      • Retry logic attempts reconnection a limited number of times before failing gracefully.
      • Logs provide clear insights into Kafka connection issues.

       

      Show
        Errors during Kafka producer initialization are caught and logged. Retry logic attempts reconnection a limited number of times before failing gracefully. Logs provide clear insights into Kafka connection issues.  
    • Unset
    • None
    • Access & Management Sprint 105, A&M Tech Debt Sprint Q1 2025

      https://glitchtip.devshift.net/insights/issues/3519394?project=38&cursor=cj0xJnA9MjAyNS0wMy0wOSsxMCUzQTM5JTNBMTUuMjE3Mzc5JTJCMDAlM0EwMA%3D%3D&query=is:unresolved

      Error message:
      Connect attempt to <kafka.conn.BrokerConnection object at 0x7f5e9e938370> returned error 111. Disconnecting.
       

      Currently, the Kafka producer (self.producer = KafkaProducer(**settings.KAFKA_AUTH)) lacks proper error handling and logging. To improve stability and observability, the following enhancements should be made:

      • Add Try-Catch Block:
        • Wrap the Kafka producer initialization in a try-except block to handle potential connection failures or misconfigurations.
        • Ensure appropriate error messages are logged when an exception occurs.
      • Enable Retry Logic:
        • it looks that KafkaProducer support retry logic, we need to enable it.

       

       

              rh-ee-ecasey Evan Casey
              lpichler@redhat.com Libor Pichler
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: