Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1573

Why is my mysql connector task running in two node, and make repeat data.

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Resolved at Apache
    • Affects Version/s: 0.10.0.Final
    • Fix Version/s: None
    • Component/s: docker-images
    • Labels:
      None
    • Steps to Reproduce:
      Hide

      I can't reproduce the problem.

      Show
      I can't reproduce the problem.

      Description

      Debezium cluster mode.

      Problem

      One connector task is running in two node, and make repeat data.
      At the very start, after I created tow mysql connector. All is ok, one node one task.

      However, after a period of time, my table topic of test_cloud-connector-2 receive double data. And I found both node were perfoming the task of test_cloud-connector-2.

      The attachment is the log of two nodes. (dhhqh node and 2r2qk node)

      Before 2019-10-22 08:07, test_cloud-connector-2 were running in 2r2qk node, test_cloud-connector-1 was running in dhhqh node. At this time, the task of test_cloud-connector-2 was running in 2r2qk node according to api connect.url/connectors/test_cloud-connector-2/status

      After 2019-10-22 08:07, test_cloud-connector-2 was running in the two node. However, the task was running in dhhqh node according to api connct.url/connectors/test_cloud-connector-2/status. As if, the task in the 2r2qk node was an orphan task.

      Connector Config

      // test_cloud-connector-1
      {
          "connector.class": "io.debezium.connector.mysql.MySqlConnector",
          "snapshot.locking.mode": "none",
          "database.user": "****",
          "database.server.id": "222333441",
          "tasks.max": "1",
          "database.history.kafka.bootstrap.servers": "192.168.100.228:9092",
          "database.history.kafka.topic": "base.history.test_cloud.1",
          "database.server.name": "base.server.tc1",
          "database.port": "3306",
          "table.whitelist": "test_cloud.orders",
          "snapshot.new.tables": "parallel",
          "database.hostname": "***********",
          "database.password": "*************",
          "name": "test_cloud-connector-1",
          "database.history.store.only.monitored.tables.ddl": "true",
          "database.whitelist": "test_cloud",
          "snapshot.mode": "schema_only"
      }
      
      // test_cloud-connector-2
      {
          "connector.class": "io.debezium.connector.mysql.MySqlConnector",
          "snapshot.locking.mode": "none",
          "database.user": "*******",
          "database.server.id": "222333442",
          "tasks.max": "1",
          "database.history.kafka.bootstrap.servers": "192.168.100.228:9092",
          "database.history.kafka.topic": "base.history.test_cloud.2",
          "database.server.name": "base.server.tc2",
          "database.port": "3306",
          "table.whitelist": "test_cloud.orders",
          "snapshot.new.tables": "parallel",
          "database.hostname": "***************",
          "database.password": "***************",
          "name": "test_cloud-connector-2",
          "database.history.store.only.monitored.tables.ddl": "true",
          "database.whitelist": "test_cloud",
          "snapshot.mode": "schema_only"
      }
      

      Cluster

      I have deployed my debezium to openshift. (one service, two replicas) The deployment env follows:

      - env:
                  - name: BOOTSTRAP_SERVERS
                    value: '192.168.100.228:9092'
                  - name: GROUP_ID
                    value: test-cloud
                  - name: CONFIG_STORAGE_TOPIC
                    value: base.test-cloud.config
                  - name: OFFSET_STORAGE_TOPIC
                    value: base.test-cloud.offset
                  - name: STATUS_STORAGE_TOPIC
                    value: base.test-cloud.status
                  - name: CONNECT_KEY_CONVERTER_SCHEMAS_ENABLE
                    value: 'false'
                  - name: CONNECT_VALUE_CONVERTER_SCHEMAS_ENABLE
                    value: 'false'
                  - name: CONNECT_PRODUCER_MAX_REQUEST_SIZE
                    value: '20971520'
                  - name: CONNECT_DATABASE_HISTORY_KAFKA_RECOVERY_POLL_INTERVAL_MS
                    value: '1000'
                  - name: HEAP_OPTS
                    value: '-XX:+UseContainerSupport -XX:MaxRAMPercentage=75.0'
      

        Gliffy Diagrams

          Attachments

          1. 12_31_d8c7j_1.jpg
            12_31_d8c7j_1.jpg
            524 kB
          2. 12_31_d8c7j.log
            116 kB
          3. 12_31_wmcp5.log
            379 kB
          4. 12-2r2qk.log
            39 kB
          5. 12-dhhqh.log
            82 kB

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                kai.hou kai hou
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: