Current(4.1.x) transaction implementation in CacheStores is brocken in several ways:

1st problem.

AbstractCacheStore.prepare:
   public void prepare(List<? extends Modification> mods, GlobalTransaction tx, boolean isOnePhase) throws CacheLoaderException {
      if (isOnePhase) {
         applyModifications(mods);
      } else {
         transactions.put(tx, mods);
      }
   }

If this is 1PC we apply the modifications in the prepare phase - we should do it in the commit phase (as JTA does it).

2nd problem.
This currently exhibits during commit/rollback with JdbcXyzCacheStore, but it is rather a more general cache store issue.
When using a TransactionManager, during TM.commit AbstractCacheStore.commit is being called internally which tries to apply all the modifications that happened during that transaction.
Within the scope of AbstractCacheStore.commit, JdbcStore obtains a connection from a DataSource and tries to write the modifications on that connection.
Now if the DataSource is managed (e.g. by an A.S.) on the DS.getConnection call the A.S. would try to enlist the connection with the ongoing transaction by calling Transaction.enlistResource(XAResource xaRes) [1]
This method fails with an IllegalStateException, because the transaction's status is preparing (see javax.transaction.Transaction.enlistResource).
Suggested fix:

• the modifications should be registered to the transaction as they happen(vs. during prepare/commit as it happens now)
• this requires API changes in CacheStore, e.g.
  void store(InternalCacheEntry entry)
  should become
  void store(InternalCacheEntry entry, GlobalTransaction gtx)
  (gtx would be null if this is not a transactional call).

[1] This behavior is specified by the JDBC 2.0 Standard Extension API, chapter 7 - distributed transaction