Test setup:

- Attached cluster.xml
- 4 node cluster, each on physical boxes
- Build and deploy the InfinispanQueryPerformanceCacheManager on all 4 nodes.  It will look for the cluster.xml on

{jboss.server.home.dir}/conf/bni

- The "udp.xml" JGroups configuration (from the unit test suite) should be put under:

{jboss.server.home.dir}/conf/bni/stacks

- Deploy the web service onto all 4 nodes.  The cache is peer-to-peer, so it should cluster.

Here are the test steps:

  
1.  Run the "ingestObjects" command with the following parameters:

    CacheName : cache_nojta_async
    IsSmallObject : false
    prefix : A
    ObjectNumber : 10000

(On my system, it took 20s to create all these objects).

2.  Run the service:  "getObjectsFromCache" with the following parameters:

    CacheName : cache_nojta_async
    IsSmallObject : false
    prefix : A
    From : 1
    To : 10000

This iterates through the cache and returns all the objects.  This took about 3.8s

3.  Run the service "getCacheSize" on each of the nodes.  This is where we see missing cache objects.  From the configuration, we have 3 owners.  Here was our output:

Node 1:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
   <env:Header/>
   <env:Body>
      <ns2:getCacheSizeResponse xmlns:ns2="urn:net.beaumaris.InfinispanQueryPerformanceWS.PerformanceWebService">
         <return>6407</return>
      </ns2:getCacheSizeResponse>
   </env:Body>
</env:Envelope>

Node 2:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
   <env:Header/>
   <env:Body>
      <ns2:getCacheSizeResponse xmlns:ns2="urn:net.beaumaris.InfinispanQueryPerformanceWS.PerformanceWebService">
         <return>5623</return>
      </ns2:getCacheSizeResponse>
   </env:Body>
</env:Envelope>

Node 3:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
   <env:Header/>
   <env:Body>
      <ns2:getCacheSizeResponse xmlns:ns2="urn:net.beaumaris.InfinispanQueryPerformanceWS.PerformanceWebService">
         <return>8576</return>
      </ns2:getCacheSizeResponse>
   </env:Body>
</env:Envelope>

Node 4:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
   <env:Header/>
   <env:Body>
      <ns2:getCacheSizeResponse xmlns:ns2="urn:net.beaumaris.InfinispanQueryPerformanceWS.PerformanceWebService">
         <return>9392</return>
      </ns2:getCacheSizeResponse>
   </env:Body>
</env:Envelope>

6407 + 5623 + 8576 + 9392 = 29998(?!)



4.  Now shutdown a node.   Repeatedly run "getObjectsFromCache."  Not all of the objects are available.  Why?  From the wiki, it seems like that the first successful GET should return the object.  Is that not true on a state transfer?