Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2117

Encountered error when snapshotting collection type column

    XMLWordPrintable

Details

    Description

      If the type of a column in cassandra is a collection type like list, set or map, currently the snapshot will fail because ttl() and writetime(), which are needed for calculating snapshotting information for a column, cannot be applied for a collection type.

      16:25:12.799 [pool-4-thread-4] INFO  i.d.c.cassandra.SnapshotProcessor - Executing snapshot query 'SELECT UNIXTIMESTAMPOF(NOW()) AS execution_time,"data_center","service_name","cluster","group",ttl("group") AS group_ttl,writetime("group") AS group_writetime FROM test_cdc.deployment_in_progress;' with consistency level ONE
      16:25:12.809 [pool-4-thread-4] ERROR i.d.c.c.CassandraConnectorTask - Encountered exception while running Snapshot Processor; stopping all processors in Cassandra Connector Task
      com.datastax.driver.core.exceptions.InvalidQueryException: Cannot use selection function ttl on collections
      	at com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:50) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at io.debezium.connector.cassandra.CassandraClient.execute(CassandraClient.java:103) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at io.debezium.connector.cassandra.SnapshotProcessor.takeTableSnapshot(SnapshotProcessor.java:156) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at io.debezium.connector.cassandra.SnapshotProcessor.snapshot(SnapshotProcessor.java:120) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at io.debezium.connector.cassandra.SnapshotProcessor.process(SnapshotProcessor.java:91) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      	at io.debezium.connector.cassandra.AbstractProcessor.start(AbstractProcessor.java:61) ~[debezium-connector-cassandra-1.2.0-SNAPSHOT-jar-with-dependencies.jar:1.2.0-SNAPSHOT]
      

      Attachments

        Activity

          People

            zhou.bing@husky.neu.edu Bingqin Zhou
            zhou.bing@husky.neu.edu Bingqin Zhou
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: