Details
-
Bug
-
Resolution: Done
-
Major
-
fuse-7.11.1-GA
-
None
-
False
-
None
-
False
-
%
-
Todo
-
Workaround Exists
-
-
Very Likely
Description
Due to the following known issue in JBoss Marshalling before 2.0.11.Final, there is a possibility of performance issue for camel-infinispan.
https://issues.redhat.com/browse/JDG-4351
https://issues.redhat.com/browse/JBMAR-235
—
The following is pseudo-code to reproduce the issue. Note that it's required to use the same RemoteCacheManager object and execute all the following operations in the same thread.
- PUTALL with Map object of a certain size // no performance issue - PUT with a huge size List object - PUTALL with Map object of a certain size // performance issue occurs.
To implement this pseudo-code, the reproducer uses a loop to execute 3 PUTALL operations with a Map object of a certain size, 1 PUT operation with a huge size of List object, then 3 PUTALL operations with Map object of a certain size again. For example:
public class MyRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { Map<String, String> map = new HashMap<>(); for(int i = 0; i < 3000; i++) { map.put("k" + i, "v" + i); } List<String> list = new ArrayList<>(); for(int i = 0; i < 1000000; i++) { list.add(new String("foo" + i)); } from("timer://foo?repeatCount=1") .log("----------------- JBoss Marshalling Version = " + Version.getVersionString()) .setHeader("counter", simple("1")) .loopDoWhile(simple("${header.counter} <= 7")) .choice() .when(simple("${header.counter} == '4'")) .setHeader(InfinispanConstants.OPERATION, constant(InfinispanOperation.PUT)) .setHeader(InfinispanConstants.KEY).constant("hugelist") .setHeader(InfinispanConstants.VALUE).constant(list) .log(">>> operation ${header.counter} start --- put() huge list") .otherwise() .setHeader(InfinispanConstants.OPERATION, constant(InfinispanOperation.PUTALL)) .setHeader(InfinispanConstants.MAP).constant(map) .log(">>> operation ${header.counter} start --- putAll()") .end() .to("infinispan:distributed?hosts=127.0.0.1:11222") .log("<<< operation ${header.counter} end") .setHeader("counter", simple("${header.counter}++")) .end(); } }
Also there are some "Fuse on Openshift" users, this reproducer works for "Fuse on Openshift" too.
Steps to reproduce the performance issue with the attached reproducer:
See also README.md in the attached reproducer-camel-infinispan.zip
First, set up Red Hat Data Grid (RHDG) Server:
1. Download and unzip a binary from our internal file server: http://download.devel.redhat.com/released/JBoss-middleware/datagrid/8.4.2/redhat-datagrid-8.4.2-server.zip
2.Modify server/conf/infinispan.xml to create distributed-cache named "distributed" and disable security for testing purpose
--- server/conf/infinispan.xml.BAK 2023-05-23 11:47:34.319251572 +0900
+++ server/conf/infinispan.xml 2023-07-24 08:38:13.993779347 +0900
@@ -7,9 +7,13 @@
<cache-container name="default" statistics="true">
<transport cluster="${infinispan.cluster.name:cluster}" stack="${infinispan.cluster.stack:tcp}" node-name="${infinispan.node.name:}"/>
- <security>
- <authorization/>
- </security>
+ <!-- <security> -->
+ <!-- <authorization/> -->
+ <!-- </security> -->
+ <distributed-cache-configuration name="dist-cache-template">
+ <encoding media-type="application/x-jboss-marshalling"/>
+ </distributed-cache-configuration>
+ <distributed-cache name="distributed" configuration="dist-cache-template" statistics="true"/>
</cache-container>
<server xmlns="urn:infinispan:server:14.0">
@@ -48,6 +52,7 @@
</security-realms>
</security>
- <endpoints socket-binding="default" security-realm="default" />
+ <!-- <endpoints socket-binding="default" security-realm="default" /> -->
+ <endpoints socket-binding="default"/>
</server>
</infinispan>
3. Start RHDG Server. Make sure to use JDK 11.
$ echo $JAVA_HOME /usr/lib/jvm/java-11-openjdk $ ./bin/server.sh
Then, run camel route with camel-infinispan that accesses the above RHDG Server
1. Download and unzip the attached reproducer-camel-infinispan.zip
2. Build the maven project
mvn clean install mvn karaf:assembly
3. Start the build application
$ ./target/assembly/bin/fuse
4. Check the log (console output or karaf.log).
You can see the first 3 putAll() operations (1-3) completed within 1 second, but the last 3 putAll() operations (5-7) after put() huge list took aroundabout 15-20 seconds which is very slow comparing with the first 3 operations.
2023-07-24 18:52:40,933 | INFO | Event Dispatcher: 1 | o.i.c.h.RemoteCacheManager | 127 - org.infinispan.client-hotrod - 9.4.19.Final-redhat-00001 | ISPN004021: Infinispan version: 9.4.19.Final-redhat-00001 2023-07-24 18:52:40,964 | INFO | Event Dispatcher: 1 | o.a.c.b.BlueprintCamelContext | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | Route: route1 started and consuming from: timer://foo?repeatCount=1 2023-07-24 18:52:40,967 | INFO | Event Dispatcher: 1 | o.a.c.b.BlueprintCamelContext | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | Total 1 routes, of which 1 are started 2023-07-24 18:52:40,969 | INFO | Event Dispatcher: 1 | o.a.c.b.BlueprintCamelContext | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | Apache Camel 2.23.2.fuse-7_11_1-00015-redhat-00002 (CamelContext: log-example-context) started in 1.714 seconds 2023-07-24 18:52:41,993 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | ----------------- JBoss Marshalling Version = 2.0.9.Final 2023-07-24 18:52:42,000 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | ----------------- Infinispan Version = 9.4.19.Final-redhat-00001 2023-07-24 18:52:42,018 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | >>> operation 1 start --- putAll() 2023-07-24 18:52:42,068 | INFO | lient-async-pool-1-1 | o.i.c.h.i.p.Codec21 | 127 - org.infinispan.client-hotrod - 9.4.19.Final-redhat-00001 | ISPN004006: Server sent new topology view (id=727952508, age=0) containing 1 addresses: [127.0.0.1:11222] 2023-07-24 18:52:42,264 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | <<< operation 1 end 2023-07-24 18:52:42,269 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | >>> operation 2 start --- putAll() 2023-07-24 18:52:42,316 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | <<< operation 2 end 2023-07-24 18:52:42,318 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | >>> operation 3 start --- putAll() 2023-07-24 18:52:42,367 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | <<< operation 3 end 2023-07-24 18:52:42,369 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | >>> operation 4 start --- put() huge list 2023-07-24 18:52:45,031 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | <<< operation 4 end 2023-07-24 18:52:45,033 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | >>> operation 5 start --- putAll() 2023-07-24 18:53:06,940 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | <<< operation 5 end 2023-07-24 18:53:06,942 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | >>> operation 6 start --- putAll() 2023-07-24 18:53:24,899 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | <<< operation 6 end 2023-07-24 18:53:24,902 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | >>> operation 7 start --- putAll() 2023-07-24 18:53:44,345 | INFO | ead #1 - timer://foo | route1 | 119 - org.apache.camel.camel-core - 2.23.2.fuse-7_11_1-00015-redhat-00002 | <<< operation 7 end
Attachments
Issue Links
- relates to
-
ENTESB-21755 Merge custom etc/org.apache.karaf.features.xml with the one from KAR
-
- Done
-
- links to
