Details
-
Feature Request
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
-
0
-
0%
Description
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
In some cases, there are errors when the events are being consumed from the vstream. We should use the property event.processing.failure.handling.mode for handling events properly. This is similar to how it's done for postgres.
One example of this we observed in our deploy of Debezium is when the message exceeds the max grpc size. Users may often want to skip these oversized messages since Kafka cannot handle them efficiently. Currently, enabling skip will still cause the connector to fail with this exception:
io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304: 6653790 at io.grpc.Status.asRuntimeException(Status.java:533) at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:478) at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463) at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427) at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460) at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:616) at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:69) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:802) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:781) at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829)
Implementation ideas (optional)
Add handling to the onError method of the VitessReplicationConnection to allow for reading this config property and handling errors appropriately
Attachments
Issue Links
- links to
-
RHEA-2023:120698
Red Hat build of Debezium 2.3.4 release
