Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6852

Always reset VStream grpc channel when max size is exceeded

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Major Major
    • 2.4.0.Beta2
    • None
    • vitess-connector
    • None

      Feature request or enhancement

      Which use case/requirement will be addressed by the proposed feature?

      When GRPC max size is exceeded and failure processing mode skip/warn is enabled the GRPC channel will not be resumed. We should reset and resume the VStream/GRPC channel regardless of what this setting was.

      The behavior right now would be that if skip/warn is enabled and a message is received over the size limit, then the connector would silently stop streaming data for this shard. We should always fail loudly & reset the channel since there is no skip behavior supported for grpc channels.

      Implementation ideas (optional)

      At the time of DBZ-6510 we thought this would allow us to skip messages over the size in the GRPC channel. This is not the case, and the GRPC channel is completely shut down if this message size is exceeded. So we should always reset/retry. We can adapt the current tests to ensure that we are still failing on this grpc size with the warn/skip settings. 

              Unassigned Unassigned
              tthorn Thomas Thornton
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: