Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4359

Debezium monitors a table with 48 fields in Oracle and inserts three pieces of data into the table in batch. Debezium only writes one log to topic, and two logs are not written in

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Major
    • None
    • 1.7.0.CR1
    • oracle-connector
    • None
    • False
    • False
    • Hide

      The first step:

              Three fields are inserted into the table, but only one record is written to topic

               The second step:

              When we empty the table, we find that three deleted records are written to topic, but at the same time, there are three more null records

            

       

      I almost forgot that these are the three records I recorded when I used logminer to parse redo (current) and insert data records into the table

       

       

       

       

       

       

       

       

      Show
      The first step:         Three fields are inserted into the table, but only one record is written to topic          The second step:         When we empty the table, we find that three deleted records are written to topic, but at the same time, there are three more null records          I almost forgot that these are the three records I recorded when I used logminer to parse redo (current) and insert data records into the table                

    Description

      At first, we batch inserted 20000 pieces of data into the table, and found that topic recorded only 500 pieces of data, which was thought to be Kafka's loss of data. After troubleshooting, we found that it was not Kafka's problem, because when we inserted three pieces of data into the table, debezium recorded only one log in topic, and then we thought that Oracle did not record the log of our inserted data in redo, Logminer is used to parse the redo (current) log. It is found that the redo records all the changes of data we insert into the table. The last problem points to debezium. Although debezum does not know how to deal with it after logimer parsing, it seems that it is his problem at present,

      In addition, when we execute the batch insert statements, we use the (insert into table select * from table) line format, which will cause problems. However, if we separate the batch insert statements, we use three insert into statements to insert data into the monitored table, and it is found that there will be no lack of records in topic

      Attachments

        1. cvCode.png
          cvCode.png
          15 kB
        2. cvError.png
          cvError.png
          97 kB
        3. ddl.png
          ddl.png
          15 kB
        4. debezium-connector-oracle.zip
          17.82 MB
        5. error.png
          error.png
          199 kB
        6. GMT.png
          GMT.png
          2 kB
        7. main.png
          main.png
          68 kB
        8. TZ_W_PER_TEST.sql
          3 kB

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            ryan-0526 逸南 丁 (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: