Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2656

Debezium 1.3 SQL Conector- consumer command return only initial set of records

    XMLWordPrintable

Details

    • Task
    • Resolution: Not a Bug
    • Major
    • None
    • 1.3.0.Final
    • sqlserver-connector
    • None
    • False
    • False
    • debezium 1.3.0 SQL
    • Undefined

    Description

      Standalone Kafka Windows, Debezium 1.3 SQL Connector

      my consumer command return only initial set of records, does not continuously stream data using debezium 1.3 command used ./kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic SIP.SIP.SHIPMENT --partition 0
      Config I used:
      name=SIPSHIPMENTProducer
      connector.class=io.debezium.connector.sqlserver.SqlServerConnector
      database.hostname=101.53.156.140
      database.port=1433
      database.user=read
      database.password=read
      database.dbname=SIP
      database.server.name=SIP
      table.include.list=SIP.SHIPMENT
      database.history.kafka.bootstrap.servers=localhost:9092
      database.history.kafka.topic=SIPSHIPMENT
      snapshot.isolation.mode=read_uncommitted
      Topic List:
      SIP
      SIP.SIP.SHIPMENT
      SIPSHIPMENT

      PS E:\kafka\bin\windows> ./kafka-console-consumer.bat --topic SIP.SIP.SHIPMENT --from-beginning --bootstrap-server localhost:9092
      {"schema":{"type":"struct","fields":[{"type":"struct","fields":[

      {"type":"string","optional":false,"field":"SHIPMENTID"}

      ,{"type":"string","optional":true,"field":"SHIPMENTNUMBER"}],"optional":true,"name":"SIP.SIP.SHIPMENT.Value","field":"before"},{"type":"struct","fields":[

      {"type":"string","optional":false,"field":"SHIPMENTID"}

      ,{"type":"string","optional":true,"field":"SHIPMENTNUMBER"}],"optional":true,"name":"SIP.SIP.SHIPMENT.Value","field":"after"},{"type":"struct","fields":[

      {"type":"string","optional":false,"field":"version"}

      ,{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":

      {"allowed":"true,last,false"}

      ,"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"change_lsn"},{"type":"string","optional":true,"field":"commit_lsn"},{"type":"int64","optional":true,"field":"event_serial_no"}],"optional":false,"name":"io.debezium.connector.sqlserver.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[

      {"type":"string","optional":false,"field":"id"}

      ,{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"SIP.SIP.SHIPMENT.Envelope"},"payload":{"before":null,"after":

      {"SHIPMENTID":"1","SHIPMENTNUMBER":"1"}

      ,"source":{"version":"1.3.0.Final","connector":"sqlserver","name":"SIP","ts_ms":1602646778416,"snapshot":"last","db":"SIP","schema":"SIP","table":"SHIPMENT","change_lsn":null,"commit_lsn":"0000002b:000020e5:0001","event_serial_no":null},"op":"r","ts_ms":1602646778419,"transaction":null}}

      Log files and contig files attached 

      CDC image copied too

       

       

      Attachments

        1. Capturecdc.PNG
          Capturecdc.PNG
          10 kB
        2. connect-distributed.properties
          5 kB
        3. connect-log4j.properties
          2 kB
        4. log4j.properties
          5 kB
        5. logs.zip
          363 kB
        6. server.zip
          5 kB
        7. tmp.zip
          1.15 MB

        Activity

          People

            Unassigned Unassigned
            sneevand navin d (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: