-
Task
-
Resolution: Done
-
Normal
-
None
-
None
-
None
-
5
-
False
-
None
-
False
-
-
When kafka reboots, we see a large number of GlitchTip alerts. Those are caused by our check_kafka_connection method which iterates over all the brokers and creates a new BrokerConnection to check the connection. Given the number of brokers and kafka listeners we have, and frequency of liveness checks (15 seconds), and that we are creating a NEW brokerconnection for each broker for every status check, we see a lot of alerts.
Additionally, this BrokerConnection comes from kafka-python which was last updated in 2020, and this is the only thing we use it for.
We should update our kafka checks to utilize something else so we don't see such a large frequency of alerts when kafka is restarting.
