Bug report

What Debezium connector do you use and what version?

1.9.6

What is the connector configuration?

"connector.class": "io.debezium.connector.mysql.MySqlConnector", 
"value.converter": "org.apache.kafka.connect.json.JsonConverter", 
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"database.hostname": "$HOSTNAME",
"database.password": "****************", 
"database.port": "$PORT", 
"database.server.name": "dbserver", 
"database.user": "kafkauser", 
"table.include.list": "db.orders,db.users", 
"snapshot.mode": "initial", 
"tasks.max": "1"

What is the captured database version and mode of depoyment?

Percona 8.0.30

What behaviour do you expect?

Connector should correctly determine that the binlog position to start from is available or not. If not available, go for re-snapshotting if "snapshot.mode":"when_needed" else throw an exception to configure the snapshot.mode to when_needed.

What behaviour do you see?

The connector logic concludes that the binlog position is available and later fails with the following exception:

io.debezium.DebeziumException: Could not find first log file name in binary log index file Error code: 1236; SQLSTATE: HY000.
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1194)
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1239)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
	at java.base/java.lang.Thread.run(Thread.java:831)
Caused by: com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1043)
	... 3 more

Do you see the same behaviour using the latest relesead Debezium version?

Not tested. This happened for a fully managed connector running in Confluent Cloud. Looks like a corner case was missing in the code and is still present in the latest version.

Do you have the connector logs, ideally from start till finish?

Following are some important log lines after a connector failure followed by a re-start that uncovered the corner case:

//the GTID set from the offset is exactly same as the GTID set present in the server 

MySQL current GTID set 07db89ec-40c9-11ed-af00-90e2baac0d70:1-16,1130e86e-22ea-11ed-87a6-90e2baac0d70:1-283,4dd91a5d-0c28-11ed-a4d8-78ac440f46a8:1-1615955779,6ca752da-3da9-11ed-8f1d-42010a3520fc:1-294165328,7dd411a6-1836-11ed-aa02-00505602075e:1-286392799,81f6a2a6-4f24-11ed-aed9-b8cef6069aab:1-5586,afa1b650-4115-11ed-a86b-42010a3501db:1-1480765156,c11db438-d1d3-11ec-a908-42010a352021:1-9202553,dd272b1d-b1b3-11ec-a210-f8f21eb83da0:1-2869705,fb3c5af9-4361-11ed-85be-90e2baac0d70:1-103 does contain the GTID set required by the connector 07db89ec-40c9-11ed-af00-90e2baac0d70:1-16,1130e86e-22ea-11ed-87a6-90e2baac0d70:1-283,4dd91a5d-0c28-11ed-a4d8-78ac440f46a8:1-1615955779,6ca752da-3da9-11ed-8f1d-42010a3520fc:1-294165328,7dd411a6-1836-11ed-aa02-00505602075e:1-286392799,81f6a2a6-4f24-11ed-aed9-b8cef6069aab:1-5586,afa1b650-4115-11ed-a86b-42010a3501db:1-1480765156,c11db438-d1d3-11ec-a908-42010a352021:1-9202553,dd272b1d-b1b3-11ec-a210-f8f21eb83da0:1-2869705,fb3c5af9-4361-11ed-85be-90e2baac0d70:1-103 

//the purged GTID set is also exactly the same as the previous sets 

Server has already purged 07db89ec-40c9-11ed-af00-90e2baac0d70:1-16,1130e86e-22ea-11ed-87a6-90e2baac0d70:1-283,4dd91a5d-0c28-11ed-a4d8-78ac440f46a8:1-1615955779,6ca752da-3da9-11ed-8f1d-42010a3520fc:1-294165328,7dd411a6-1836-11ed-aa02-00505602075e:1-286392799,81f6a2a6-4f24-11ed-aed9-b8cef6069aab:1-5586,afa1b650-4115-11ed-a86b-42010a3501db:1-1480765156,c11db438-d1d3-11ec-a908-42010a352021:1-9202553,dd272b1d-b1b3-11ec-a210-f8f21eb83da0:1-2869705,fb3c5af9-4361-11ed-85be-90e2baac0d70:1-103 

/** 
inside the isBinlogAvailable() method, the gtidSetToReplicate and nonPurgedGtidSetToReplicate are empty and hence equal. So the following condition becomes false: 
if (!gtidSetToReplicate.equals(nonPurgedGtidSetToReplicate)) 

And hence, the isBinlogAvailable() method returns true. The connector then tries to read the binlog file and fails with this exception:
com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file 
**/

How to reproduce the issue using our tutorial deployment?

Not sure how the GTID set in the server is identical to the purged GTID set. So not able to reproduce.

Feature request or enhancement

For feature requests or enhancements, provide this information, please:

Which use case/requirement will be addressed by the proposed feature?

Implementation ideas (optional)

Implementation ideas (optional) Update the condition to also check if the gtidSetToReplicate is empty. If yes, then the method isBinlogAvailable() should return false.

Details

Description