Loading...

Type: Bug
Resolution: Unresolved
Priority: Blocker
Fix Version/s: 35.0.1.Final
Affects Version/s: 35.0.0.Final
Component/s: IIOP, Transactions
Labels:
None

Steps to Reproduce:
Hide

Steps to reproduce:

Download the latest Wildfly 35.0.0.Final (or built from WildFly main branch) and unzip it. Set JBOSS_HOME env variable to unzipped directory.

git clone -b master git@gitlab.cee.redhat.com:jbossqe-eap/tests-transactions.git

cd tests-transactions

mvn clean verify -Dtest=TxTimeoutJMSCrashRecoveryTestCase#txnTimeoutClientDuringPrepare -Dsurefire.failIfNoSpecifiedTests=false -Djboss.dist=${JBOSS_HOME} -Dskip-download-sources -U --batch-mode -Dmaven.test.failure.ignore=true -Dsurefire.test.failure.ignore=true --fail-never -Dversion.server=35.0.0.Final -Dts.timeout.factor=150 -Djbossts.noJTA

Test fails with error:

java.lang.AssertionError: Number of known instances of org.jboss.as.test.jbossts.common.TestXAResource does not match expected:<1> but was:<0>

however by checking server log: ./jbossts/target/jbossas-jbossts/standalone/log/TxTimeoutJMSCrashRecoveryTestCase_txnTimeoutClientDuringPrepare_jts_server.log you can see the java.lang.OutOfMemoryError.
Show
Steps to reproduce: Download the latest Wildfly 35.0.0.Final (or built from WildFly main branch) and unzip it. Set JBOSS_HOME env variable to unzipped directory. git clone -b master git@gitlab.cee.redhat.com:jbossqe-eap/tests-transactions.git cd tests-transactions mvn clean verify -Dtest=TxTimeoutJMSCrashRecoveryTestCase#txnTimeoutClientDuringPrepare -Dsurefire.failIfNoSpecifiedTests=false -Djboss.dist=${JBOSS_HOME} -Dskip-download-sources -U --batch-mode -Dmaven.test.failure.ignore=true -Dsurefire.test.failure.ignore=true --fail-never -Dversion.server=35.0.0.Final -Dts.timeout.factor=150 -Djbossts.noJTA Test fails with error: java.lang.AssertionError: Number of known instances of org.jboss.as.test.jbossts.common.TestXAResource does not match expected:<1> but was:<0> however by checking server log: ./jbossts/target/jbossas-jbossts/standalone/log/TxTimeoutJMSCrashRecoveryTestCase_txnTimeoutClientDuringPrepare_jts_server.log you can see the java.lang.OutOfMemoryError.
[QE] How to address?:
---
[QE] Why QE missed?:
---

There is OOME failure in of the JTS test scenarios which occurs only on JDK 21 (it passes on JDK 11/17).

Test Scenario: Test of transaction timeouting on client during prepare when the prepare phase was not fully processed:

enlist xa resource on client
call to second server
enlist test xa on server
enlist xa resource on server
enlist test xa on client
waiting during prepare phase to txn timeout is hit - wait is established on client
finish prepare phase
commit all participants

Expected Result:
XA transaction is commited. There are no unexpected WARN/ERROR logs.

Actual Result:
Test fails as there is OOME thrown on the server:

2025-01-14 16:02:53,592 WARNING [javax.enterprise.resource.corba._DEFAULT_.rpc.encoding] (p: default-threadpool; w: Idle) "IOP00810210: (MARSHAL) Error from readValue on ValueHandler in CDRInputStream": org.omg.CORBA.MARSHAL:   vmcid: SUN  minor code: 210 completed: Maybe
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.logging.ORBUtilSystemException.valuehandlerReadError(ORBUtilSystemException.java:6976)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:1013)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:844)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_abstract_interface(CDRInputStream_1_0.java:838)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_abstract_interface(CDRInputStream_1_0.java:828)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream.read_abstract_interface(CDRInputStream.java:287)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.readObjectDelegate(IIOPInputStream.java:302)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.readObjectOverride(IIOPInputStream.java:464)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:531)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:498)
        at java.base/java.util.HashSet.readObject(HashSet.java:353)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:326)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.invokeObjectReader(IIOPInputStream.java:1638)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.inputObject(IIOPInputStream.java:1161)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.simpleReadObject(IIOPInputStream.java:338)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ValueHandlerImpl.readValueInternal(ValueHandlerImpl.java:341)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ValueHandlerImpl.readValue(ValueHandlerImpl.java:307)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:999)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream.read_value(CDRInputStream.java:271)
        at org.wildfly.iiop-openjdk@36.0.0.Beta1-SNAPSHOT//org.wildfly.iiop.openjdk.rmi.marshal.CDRStream$ValuetypeReader.read(CDRStream.java:640)
        at org.wildfly.iiop-openjdk@36.0.0.Beta1-SNAPSHOT//org.wildfly.iiop.openjdk.rmi.marshal.strategy.SkeletonStrategy.readParams(SkeletonStrategy.java:107)
        at org.jboss.as.ejb3@36.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.iiop.EjbCorbaServant._invoke(EjbCorbaServant.java:271)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.protocol.CorbaServerRequestDispatcherImpl.dispatchToServant(CorbaServerRequestDispatcherImpl.java:654)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.protocol.CorbaServerRequestDispatcherImpl.dispatch(CorbaServerRequestDispatcherImpl.java:205)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.protocol.CorbaMessageMediatorImpl.handleRequestRequest(CorbaMessageMediatorImpl.java:1700)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.protocol.CorbaMessageMediatorImpl.handleRequest(CorbaMessageMediatorImpl.java:1558)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.protocol.CorbaMessageMediatorImpl.handleInput(CorbaMessageMediatorImpl.java:940)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.protocol.giopmsgheaders.RequestMessage_1_2.callback(RequestMessage_1_2.java:198)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.protocol.CorbaMessageMediatorImpl.handleRequest(CorbaMessageMediatorImpl.java:712)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.dispatch(SocketOrChannelConnectionImpl.java:474)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.doWork(SocketOrChannelConnectionImpl.java:1237)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:490)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:519)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.getConvertedChars(CDRInputStream_1_0.java:2230)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.internalReadString(CDRInputStream_1_0.java:580)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.readStringOrIndirection(CDRInputStream_1_0.java:567)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_codebase_URL(CDRInputStream_1_0.java:1729)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:937)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream.read_value(CDRInputStream.java:271)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.inputObject(IIOPInputStream.java:944)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.simpleReadObject(IIOPInputStream.java:338)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ValueHandlerImpl.readValueInternal(ValueHandlerImpl.java:341)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ValueHandlerImpl.readValue(ValueHandlerImpl.java:307)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:999)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:844)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_abstract_interface(CDRInputStream_1_0.java:838)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_abstract_interface(CDRInputStream_1_0.java:828)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream.read_abstract_interface(CDRInputStream.java:287)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.readObjectDelegate(IIOPInputStream.java:302)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.readObjectOverride(IIOPInputStream.java:464)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:531)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:498)
        at java.base/java.util.HashSet.readObject(HashSet.java:353)
        at java.base/java.lang.invoke.LambdaForm$DMH/0x00007f0d34013c00.invokeSpecial(LambdaForm$DMH)
        at java.base/java.lang.invoke.LambdaForm$MH/0x00007f0d34004400.invoke(LambdaForm$MH)
        at java.base/java.lang.invoke.LambdaForm$MH/0x00007f0d34db4000.invoke_MT(LambdaForm$MH)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:326)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.invokeObjectReader(IIOPInputStream.java:1638)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.inputObject(IIOPInputStream.java:1161)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.IIOPInputStream.simpleReadObject(IIOPInputStream.java:338)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ValueHandlerImpl.readValueInternal(ValueHandlerImpl.java:341)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.io.ValueHandlerImpl.readValue(ValueHandlerImpl.java:307)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:999)
        at javax.orb.api@10.1.0.Final//com.sun.corba.se.impl.encoding.CDRInputStream.read_value(CDRInputStream.java:271)
        at org.wildfly.iiop-openjdk@36.0.0.Beta1-SNAPSHOT//org.wildfly.iiop.openjdk.rmi.marshal.CDRStream$ValuetypeReader.read(CDRStream.java:640)

The code throwing OOME:
https://github.com/jboss/openjdk-orb/blob/main/src/share/classes/com/sun/corba/se/impl/encoding/CDRInputStream_1_0.java#L2230

suggests that it tries to create huge byte array which causes OOME in:

byte[] bytes = new byte[numBytes];

The numBytes must be huge as created heap dump has only ~60MB (Heap Size is set to 2GB so it won't be leak) and OOME happens when allocating this bytearray causing immediete OOME. numBytes is passed as parameter of the method and calculated in method:
https://github.com/jboss/openjdk-orb/blob/main/src/share/classes/com/sun/corba/se/impl/encoding/CDRInputStream_1_0.java#L491

based on bytes from InputStream. So it seems that recent changes in JDK 21 has changed the way of the serialization/deserialization and current logic bytes -> int does not work anymore.

is cloned by

JBEAP-28863 OutOfMemoryError in CDRInputStream_1_0 Due to Improper Byte Array Size Calculation on JDK 21

Coding In Progress

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates