-
Bug
-
Resolution: Done
-
Major
-
1.1.1.Final
-
None
I've started initial sync for all collections except a few blacklisted ones.
At some point task fails with the following exception:
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.lambda$null$0(MongoDbSnapshotChangeEventSource.java:126) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.kafka.connect.errors.ConnectException: Error while attempting to sync 'rs0.paler_service_sandbox.system.profile' at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.lambda$establishConnectionToPrimary$3(MongoDbSnapshotChangeEventSource.java:235) at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.executeBlocking(ConnectionContext.java:346) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.createDataEventsForCollection(MongoDbSnapshotChangeEventSource.java:439) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.createDataEventsForReplicaSet(MongoDbSnapshotChangeEventSource.java:419) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.createDataEvents(MongoDbSnapshotChangeEventSource.java:318) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.snapshotReplicaSet(MongoDbSnapshotChangeEventSource.java:217) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.lambda$null$0(MongoDbSnapshotChangeEventSource.java:115) ... 5 more Caused by: java.lang.NullPointerException at io.debezium.pipeline.AbstractChangeRecordEmitter.emitChangeRecords(AbstractChangeRecordEmitter.java:33) at io.debezium.pipeline.EventDispatcher.dispatchSnapshotEvent(EventDispatcher.java:123) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.lambda$createDataEventsForCollection$8(MongoDbSnapshotChangeEventSource.java:463) at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.executeBlocking(ConnectionContext.java:339) ... 10 more
I don't have the record itself which caused the issue.
However, I found another JIRA with the similar symptoms:
https://issues.redhat.com/browse/DBZ-1464
Looking at the code:
public class MongoDbChangeRecordEmitter { ... // it could return null here protected Operation getOperation() { if (oplogEvent.getString("op") == null) { return Operation.READ; } return OPERATION_LITERALS.get(oplogEvent.getString("op")); }
public abstract class AbstractChangeRecordEmitter { ... public void emitChangeRecords(DataCollectionSchema schema, Receiver receiver) throws InterruptedException { Operation operation = getOperation(); // operation could be null and switch will throw NPE switch (operation) {
I'll do a bit of a digging with DEBUG logs enabled to find out what is the record payload.