Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-94

Out of Memory when taking snapshot of large table

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • 0.2.4, 0.3
    • 0.2.3
    • mysql-connector
    • None

      using debezium/connect:0.2 to source a large database
      Debezium throwed OOM when taking snapshot

      Running on Amazon EC2 with 16G memory

      2016-08-04 13:12:43,901 INFO   MySQL|mysql_host_52_70_200_92_port_3307|snapshot  Step 8.2: scanned table 'gpay.app_billing' in 00:01:25.56   [io.debezium.connector.mysql.SnapshotReader]
      2016-08-04 13:14:00,793 ERROR  MySQL|mysql_host_52_70_200_92_port_3307|snapshot  Failed due to error: Aborting snapshot after running 'SELECT * FROM gpay.app_installation': Java heap space   [io.debezium.connector.mysql.SnapshotReader]
      java.lang.OutOfMemoryError: Java heap space
      	at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2205)
      	at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1984)
      	at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3403)
      	at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:470)
      	at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3105)
      	at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2336)
      	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2729)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2545)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2503)
      	at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1369)
      	at io.debezium.jdbc.JdbcConnection.query(JdbcConnection.java:277)
      	at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:311)
      	at io.debezium.connector.mysql.SnapshotReader$$Lambda$82/1145170159.run(Unknown Source)
      	at java.lang.Thread.run(Thread.java:745)
      2016-08-04 13:14:01,179 ERROR  MySQL|mysql_host_52_70_200_92_port_3307|task  Task gpay-0 threw an uncaught and unrecoverable exception   [org.apache.kafka.connect.runtime.WorkerSourceTask]
      2016-08-04 13:14:01,180 ERROR  MySQL|mysql_host_52_70_200_92_port_3307|task  Task is being killed and will not recover until manually restarted:   [org.apache.kafka.connect.runtime.WorkerSourceTask]
      org.apache.kafka.connect.errors.ConnectException: Aborting snapshot after running 'SELECT * FROM gpay.app_installation': Java heap space
      	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:118)
      	at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:388)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.OutOfMemoryError: Java heap space
      	at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2205)
      	at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1984)
      	at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3403)
      	at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:470)
      	at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3105)
      	at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2336)
      	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2729)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2545)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2503)
      	at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1369)
      	at io.debezium.jdbc.JdbcConnection.query(JdbcConnection.java:277)
      	at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:311)
      	at io.debezium.connector.mysql.SnapshotReader$$Lambda$82/1145170159.run(Unknown Source)
      	... 1 more
      

      table info:
      Name: app_installation
      Engine: InnoDB
      Version: 10
      Row_format: Dynamic
      Rows: 28267215
      Avg_row_length: 104
      Data_length: 2967470080
      Max_data_length: 0
      Index_length: 4715446272
      Data_free: 4194304
      Auto_increment: 29442980
      Create_time: 2016-08-04 05:07:17
      Update_time: NULL
      Check_time: NULL
      Collation: utf8_general_ci
      Checksum: NULL
      Create_options:
      Comment:

              rhauch Randall Hauch (Inactive)
              wangshao_jira barten barten (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: