Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5084

Cursor fetch is used for all results during connection

    XMLWordPrintable

Details

    • False
    • None
    • False

    Description

      Currently cursor fetch is used per connection (when connection is created in MySqlConnectorTask). If the cursor fetch should be used is determined by user setting - if snapshot.fetch.size is set, cursor fetch is used, otherwise not.
      If the cursor fetch is used we use binary protocol field reader, otherwise we use text protocol field reader.

      It's correct that MySQL use binary protocol for cursor fetch. It's also used for bulk reads. It seems that in the rest of the cases text based protocol is used (I wasn't able to find how isBinaryEncoded is set when the read is called).

      However, MySQL doesn't set cursor fetch per connection, it's determined based on the query/result set. Therefore within single connection MySQL can use different protocols for the different queries and as a result we may fail to decode the value. This happens e.g. during snapshot when we search for the maximum key (single result - text protocol is used) and then fetch the tables - if snapshot.fetch.size is set binary protocol is used and we fail to read maximum key as we use binary field reader (see also DBZ-4939).

      To fix it bigger refactoring is needed - we have to determine field reader based on the result set and don't use one value during whole connection, based on the snapshot.fetch.size. Another issue is how to determine result set protocol reliably. It seems that MySQL connector doesn't provide any way how to determine it.

      Attachments

        Activity

          People

            Unassigned Unassigned
            vjuranek@redhat.com Vojtech Juranek
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: