Details

      Description

      It would be very nice is we have SQLite support for infinispan. SQLite is a powerful database supporting terabyte sized databases in a file with competitive performance.

      I tried to use it as a JDBC store but the best driver I find in the internet (xerial sqlite jdbc driver) does not implement full jdbc specification and trying to use it results in exceptions.

      I think that perhaps using the non-jdbc wrapper sqlite4java may make sense for infinispan because:
      1. it promises better performance
      2. it allows using the sqlite library from OS (xerial driver uses a customized build of sqlite)

      FYI here is how I setup sqlite for infinispan (unsuccessfully):

      jboss as cli commands:
      /subsystem=datasources/jdbc-driver=sqlite:add(driver-name="sqlite",driver-module-name="org.xerial",driver-class-name=org.sqlite.JDBC)
      data-source add --name=SQLiteDS --connection-url="jdbc:sqlite:${sqlite.database.string}" --jndi-name=java:jboss/datasources/SQLiteDS --driver-name="sqlite"
      /subsystem=datasources/data-source=SQLiteDS/connection-properties=journal_mode:add(value="WAL")
      /subsystem=datasources/data-source=SQLiteDS:enable
      
      JBoss AS module definition (modules/org/xerial/main/module.xml):
      <?xml version="1.0" encoding="UTF-8"?>
      <module xmlns="urn:jboss:module:1.0" name="org.xerial">
      	<resources>
      		<resource-root path="sqlite-jdbc.jar" />
      	</resources>
      	<dependencies>
      		<module name="javax.api" />
      		<module name="javax.transaction.api"/>
      	</dependencies>
      </module>
      
      cache store/loader configuration snippet:
               <stringKeyedJdbcStore xmlns="urn:infinispan:config:jdbc:5.2" fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false" key2StringMapper="com.jboss.datagrid.chunchun.util.TwoWayKey2StringChunchunMapper">
                  <dataSource jndiUrl="java:jboss/datasources/SQLiteDS" />
                  <stringKeyedTable dropOnExit="false" createOnStart="true" prefix="ispn">
                     <idColumn name="ID_COLUMN" type="VARCHAR(255)" />
                     <dataColumn name="DATA_COLUMN" type="BLOB" />
                     <timestampColumn name="TIMESTAMP_COLUMN" type="BIGINT" />
                  </stringKeyedTable>
               </stringKeyedJdbcStore>
            </loaders>
      

      sql driver needs to be copied in the same directory as module.xml

      UPDATE: the exception is fixed with latest dev code of xerial jdbc driver, please look at comments to see remaining problems.

      The Exception I'm getting is:

      12:53:10,683 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (MSC service thread 1-3) ISPN000136: Execution error: org.infinispan.loaders.CacheLoaderException: Error while storing string key to database; key: 'user41', buffer size of value: 4918 bytes
      	at org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore.storeLockSafe(JdbcStringBasedCacheStore.java:253) [infinispan-cachestore-jdbc-5.2.5.Final.jar:5.2.5.Final]
      ...
      Caused by: java.sql.SQLException: not implemented by SQLite JDBC driver
      	at org.sqlite.Unused.unused(Unused.java:29) [sqlite-jdbc-3.7.2.jar:]
      	at org.sqlite.Unused.setBinaryStream(Unused.java:60) [sqlite-jdbc-3.7.2.jar:]
      	at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.setBinaryStream(WrappedPreparedStatement.java:871)
      	at org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore.storeLockSafe(JdbcStringBasedCacheStore.java:247) [infinispan-cachestore-jdbc-5.2.5.Final.jar:5.2.5.Final]
      	... 73 more

      The driver does not support setBinaryStream(), only setBytes(). Not sure if there are any other methods required by infinispan but not implemented.

      As a simple comparison between JDBC and direct storage, I tried an app that caches 3000 records of around 5k and 60000 records of around 0.5k (total of less than 60MiB). Bdbje store operation completes in less than a minute. With a local mysql server it takes 10 minutes. And this is on a machine with plenty of CPU and memory over an SSD. Unfortunately bdbje does not work clustered for me (ISPN-2968).
      So my point is that a local disk based, fast, reliable, transactional engine is highly needed.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                akostadinov Aleksandar Kostadinov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: