2014-01-25 08:06:57,923: Task [client1 434]: starting command: /usr/local/jdk1.7.0_45/bin/java -classpath dist/narayana-full-6.0.0.Alpha1-SNAPSHOT/etc/:dist/narayana-full-6.0.0.Alpha1-SNAPSHOT/lib/*:dist/narayana-full-6.0.0.Alpha1-SNAPSHOT/lib/ext/*:dist/narayana-full-6.0.0.Alpha1-SNAPSHOT/jacorb/lib/*:dist/narayana-full-6.0.0.Alpha1-SNAPSHOT/jacorb/etc/:ext/fscontext.jar:ext/providerutil.jar:ext/jboss-profiler-jvmti.jar:ext/jboss-logging-spi.jar:tests/build/classes/:dbdrivers/selected_dbdriver/*:dbdrivers/DB2_v9.7/db2jcc.jar:dbdrivers/jConnect-6_0/classes/jconn3.jar:dbdrivers/mssql2005_sqljdbc_2.0/enu/sqljdbc4.jar:dbdrivers/mysql-connector-java-5.1.8-bin.jar:dbdrivers/oracle_10_2_0_4/ojdbc14.jar:dbdrivers/postgresql-8.3-605.jdbc4.jar:ext/netty.jar -Dqa.debug=true -Djava.naming.provider.url=file:///tmp -Djava.naming.factory.initial=com.sun.jndi.fscontext.RefFSContextFactory -Dperformanceprofilestore.dir=config/perf_profiles/ -Djdbcprofilestore.dir=config/jdbc_profiles -Dmemorytestprofilestore.dir=config/memory_profiles/ -Dots.server.bindname=value_1 -DCoordinatorEnvironmentBean.maintainHeuristics=NO -DRecoveryEnvironmentBean.recoveryBackoffPeriod=5 -DCoreEnvironmentBean.timeoutFactor=1 -DCoordinatorEnvironmentBean.defaultTimeout=120 -DCoordinatorEnvironmentBean.dynamic1PC=false -Demma.coverage.out.file=./testoutput/crashrecovery05_2/CrashRecovery05_2_Test024/client1-coverage.ec -DportOffsetId=3 -DObjectStoreBaseDir=/home/hudson/workspace/narayana-dualstack/qa/testoutput/crashrecovery05_2/CrashRecovery05_2_Test024/client -DRecoveryEnvironmentBean.recoveryListener=true org.jboss.jbossts.qa.CrashRecovery05Clients2.Client24a value_3 value_4 
2014-01-25 08:06:58,304 err: log4j:WARN No appenders could be found for logger (org.jboss.logging).
2014-01-25 08:06:58,304 err: log4j:WARN Please initialize the log4j system properly.
2014-01-25 08:06:58,305 err: log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
2014-01-25 08:06:58,383 err: Jan 25, 2014 8:06:58 AM org.jacorb.config.JacORBConfiguration init
2014-01-25 08:06:58,383 err: INFO: base configuration loaded from classpath orb.properties
2014-01-25 08:06:58,384 err: Jan 25, 2014 8:06:58 AM org.jacorb.config.JacORBConfiguration init
2014-01-25 08:06:58,384 err: WARNING: File e:/JacORB_CONFIG_FEATURE/etc/jacorb.properties for configuration jacorb not found
2014-01-25 08:06:58,386 err: Jan 25, 2014 8:06:58 AM org.jacorb.config.JacORBConfiguration init
2014-01-25 08:06:58,387 err: INFO: configuration jacorb loaded from classpath
2014-01-25 08:06:58,395 err: 2014-01-25 08:06:58.395 FINE jacorb.interop.strict_check_on_tc_creation set to true
2014-01-25 08:06:58,395 err: 2014-01-25 08:06:58.395 INFO 
2014-01-25 08:06:58,395 err: 	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2014-01-25 08:06:58,395 err: 	JacORB V 2.3.1 (JBoss patch01), www.jacorb.org
2014-01-25 08:06:58,395 err: 	(C) The JacORB project 29-Jul-2009
2014-01-25 08:06:58,395 err: 	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2014-01-25 08:06:58,399 err: 2014-01-25 08:06:58.399 FINE RequestProcessorPoolFactory settings: thread_pool_min=5 thread_pool_max=20 thread_pool_shared=false
2014-01-25 08:06:58,401 err: 2014-01-25 08:06:58.401 FINE added ORBInitializer: org.jacorb.orb.standardInterceptors.IORInterceptorInitializer
2014-01-25 08:06:58,409 err: 2014-01-25 08:06:58.409 FINE added ORBInitializer: com.arjuna.ats.jts.orbspecific.jacorb.interceptors.interposition.InterpositionORBInitializerImpl
2014-01-25 08:06:58,429 err: 2014-01-25 08:06:58.429 FINE defaulting to DefaultSocketFactory
2014-01-25 08:06:58,443 err: 2014-01-25 08:06:58.443 FINE Maximum connection threads: 1000
2014-01-25 08:06:58,443 err: 2014-01-25 08:06:58.443 FINE Maximum idle threads: 5
2014-01-25 08:06:58,451 err: 2014-01-25 08:06:58.450 INFO Property "jacorb.hashtable_class" is set to: java.util.Hashtable
2014-01-25 08:06:58,483 err: 2014-01-25 08:06:58.483 FINE TCS set to UTF8
2014-01-25 08:06:58,496 err: 2014-01-25 08:06:58.496 INFO InterceptorManager started with 1 Server Interceptors, 1 Client Interceptors and 1 IOR Interceptors
2014-01-25 08:06:58,606 err: 2014-01-25 08:06:58.606 FINE POA RootPOA ready
2014-01-25 08:06:58,606 err: 2014-01-25 08:06:58.606 FINE Maximum connection threads: 1000
2014-01-25 08:06:58,607 err: 2014-01-25 08:06:58.607 FINE Maximum idle threads: 5
2014-01-25 08:06:58,609 err: 2014-01-25 08:06:58.607 FINE waiting for queue
2014-01-25 08:06:58,621 err: 2014-01-25 08:06:58.621 FINE created org.omg.ETF.Factories: org.jacorb.orb.iiop.IIOPFactories
2014-01-25 08:06:58,631 err: 2014-01-25 08:06:58.631 FINE created ServerSocketFactory: org.jacorb.orb.factory.DefaultServerSocketFactory
2014-01-25 08:06:58,638 err: 2014-01-25 08:06:58.638 FINE Created socket listener on 0.0.0.0/0.0.0.0:44364
2014-01-25 08:06:58,647 err: 2014-01-25 08:06:58.647 FINE reset a previous completion call
2014-01-25 08:06:58,662 err: 2014-01-25 08:06:58.662 INFO base configuration loaded from classpath orb.properties
2014-01-25 08:06:58,664 err: 2014-01-25 08:06:58.664 WARNING File e:/JacORB_CONFIG_FEATURE/etc/jacorb.properties for configuration jacorb not found
2014-01-25 08:06:58,668 err: 2014-01-25 08:06:58.665 INFO configuration jacorb loaded from classpath
2014-01-25 08:06:58,669 err: 2014-01-25 08:06:58.668 FINE jacorb.interop.strict_check_on_tc_creation set to true
2014-01-25 08:06:58,669 err: 2014-01-25 08:06:58.669 INFO created ORBSingleton
2014-01-25 08:06:58,737 err: 2014-01-25 08:06:58.737 INFO ClientConnectionManager: created new ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,737 err: 2014-01-25 08:06:58.737 FINE [0/0] creating new thread
2014-01-25 08:06:58,741 err: 2014-01-25 08:06:58.741 FINE [0/1] removed idle thread (job scheduled)
2014-01-25 08:06:58,741 err: 2014-01-25 08:06:58.741 FINE ClientGIOPConnection to 172.17.131.19:41867 (752462d5): will wait until connected
2014-01-25 08:06:58,749 err: 2014-01-25 08:06:58.749 FINE findPOA: impl_name mismatch - 8275633969 != StandardImplName
2014-01-25 08:06:58,752 err: 2014-01-25 08:06:58.752 FINE trying is_a remotely
2014-01-25 08:06:58,753 err: 2014-01-25 08:06:58.753 FINE ClientGIOPConnection to 172.17.131.19:41867 (752462d5): sendMessage() -- opening transport
2014-01-25 08:06:58,753 err: 2014-01-25 08:06:58.753 FINE created SocketFactory: org.jacorb.orb.factory.DefaultSocketFactory
2014-01-25 08:06:58,753 err: 2014-01-25 08:06:58.753 FINE Trying to connect to 172.17.131.19:41867 with timeout=90000.
2014-01-25 08:06:58,758 err: 2014-01-25 08:06:58.758 INFO Connected to 172.17.131.19:41867 from local port 36120
2014-01-25 08:06:58,760 err: 2014-01-25 08:06:58.760 FINE wrote 64 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,761 err: 2014-01-25 08:06:58.761 FINE wrote 24 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,761 err: 2014-01-25 08:06:58.761 FINE wrote 56 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,761 err: 2014-01-25 08:06:58.761 FINE wrote GIOP message of size 128 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,838 err: 2014-01-25 08:06:58.836 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,838 err: 2014-01-25 08:06:58.837 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,838 err: 2014-01-25 08:06:58.837 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,841 err: 2014-01-25 08:06:58.839 FINE wrote 76 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,841 err: 2014-01-25 08:06:58.839 FINE wrote GIOP message of size 76 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,856 err: 2014-01-25 08:06:58.855 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,856 err: 2014-01-25 08:06:58.855 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,856 err: 2014-01-25 08:06:58.856 FINE read GIOP message of size 24 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,857 err: 2014-01-25 08:06:58.857 INFO ClientConnectionManager: found ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,857 err: 2014-01-25 08:06:58.857 FINE findPOA: impl_name mismatch - 8275633969 != StandardImplName
2014-01-25 08:06:58,858 err: 2014-01-25 08:06:58.858 FINE trying is_a remotely
2014-01-25 08:06:58,859 err: 2014-01-25 08:06:58.859 FINE wrote 128 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,884 err: 2014-01-25 08:06:58.883 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,884 err: 2014-01-25 08:06:58.883 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,884 err: 2014-01-25 08:06:58.883 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,889 err: 2014-01-25 08:06:58.884 FINE wrote GIOP message of size 128 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,889 err: 2014-01-25 08:06:58.884 FINE wrote 76 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,889 err: 2014-01-25 08:06:58.884 FINE wrote GIOP message of size 76 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,892 err: 2014-01-25 08:06:58.892 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,892 err: 2014-01-25 08:06:58.892 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,892 err: 2014-01-25 08:06:58.892 FINE read GIOP message of size 24 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,896 err: 2014-01-25 08:06:58.896 FINE trying is_a remotely
2014-01-25 08:06:58,896 err: 2014-01-25 08:06:58.896 FINE wrote 128 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,899 err: 2014-01-25 08:06:58.899 FINE wrote GIOP message of size 128 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,900 err: 2014-01-25 08:06:58.900 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,900 err: 2014-01-25 08:06:58.900 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,900 err: 2014-01-25 08:06:58.900 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,901 err: 2014-01-25 08:06:58.901 FINE wrote 82 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,902 err: 2014-01-25 08:06:58.902 FINE wrote GIOP message of size 82 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,913 err: 2014-01-25 08:06:58.912 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,913 err: 2014-01-25 08:06:58.912 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,913 err: 2014-01-25 08:06:58.913 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,914 err: 2014-01-25 08:06:58.914 FINE trying is_a remotely
2014-01-25 08:06:58,914 err: 2014-01-25 08:06:58.914 FINE wrote 128 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,916 err: 2014-01-25 08:06:58.916 FINE wrote GIOP message of size 128 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,917 err: 2014-01-25 08:06:58.916 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,917 err: 2014-01-25 08:06:58.916 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:06:58,917 err: 2014-01-25 08:06:58.917 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:06:58,917 err: 2014-01-25 08:06:58.917 FINE wrote 82 bytes to 172.17.131.19:41867
2014-01-25 08:06:58,919 err: 2014-01-25 08:06:58.919 FINE wrote GIOP message of size 82 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:00,615 err: 2014-01-25 08:07:00.615 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:07:00,615 err: 2014-01-25 08:07:00.615 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:07:00,615 err: 2014-01-25 08:07:00.615 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:00,624 out: Using timeout delay factor of 1
2014-01-25 08:07:10,625 err: 2014-01-25 08:07:10.625 FINE trying is_a remotely
2014-01-25 08:07:10,626 err: 2014-01-25 08:07:10.625 FINE wrote 128 bytes to 172.17.131.19:41867
2014-01-25 08:07:10,630 err: 2014-01-25 08:07:10.629 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,630 err: 2014-01-25 08:07:10.629 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,630 err: 2014-01-25 08:07:10.629 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,630 err: 2014-01-25 08:07:10.630 FINE wrote GIOP message of size 128 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,630 err: 2014-01-25 08:07:10.630 FINE wrote 84 bytes to 172.17.131.19:41867
2014-01-25 08:07:10,632 err: 2014-01-25 08:07:10.632 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,632 err: 2014-01-25 08:07:10.632 FINE read 16 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,632 err: 2014-01-25 08:07:10.632 FINE read GIOP message of size 28 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,634 err: 2014-01-25 08:07:10.634 FINE wrote GIOP message of size 84 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,636 err: 2014-01-25 08:07:10.636 FINE trying is_a remotely
2014-01-25 08:07:10,636 err: 2014-01-25 08:07:10.636 FINE wrote 128 bytes to 172.17.131.19:41867
2014-01-25 08:07:10,638 err: 2014-01-25 08:07:10.638 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,638 err: 2014-01-25 08:07:10.638 FINE read 13 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,638 err: 2014-01-25 08:07:10.638 FINE read GIOP message of size 25 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,638 err: 2014-01-25 08:07:10.638 FINE wrote GIOP message of size 128 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,639 err: 2014-01-25 08:07:10.639 FINE wrote 84 bytes to 172.17.131.19:41867
2014-01-25 08:07:10,640 err: 2014-01-25 08:07:10.640 FINE read 12 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,640 err: 2014-01-25 08:07:10.640 FINE read 16 bytes from 172.17.131.19:41867
2014-01-25 08:07:10,640 err: 2014-01-25 08:07:10.640 FINE read GIOP message of size 28 from ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,645 err: 2014-01-25 08:07:10.641 FINE wrote GIOP message of size 84 to ClientGIOPConnection to 172.17.131.19:41867 (752462d5)
2014-01-25 08:07:10,645 out: Failed
2014-01-25 08:07:10,645 err: 2014-01-25 08:07:10.645 FINE POA RootPOAshutdown is in progress
2014-01-25 08:07:10,647 err: 2014-01-25 08:07:10.647 FINE POA RootPOAdestruction is apparent
2014-01-25 08:07:10,647 err: 2014-01-25 08:07:10.647 FINE POA RootPOAclear up the queue ...
2014-01-25 08:07:10,647 err: 2014-01-25 08:07:10.647 FINE POA RootPOA... done
2014-01-25 08:07:10,647 err: 2014-01-25 08:07:10.647 FINE POA RootPOAremove all processors from the pool ...
2014-01-25 08:07:10,648 err: 2014-01-25 08:07:10.647 FINE POA RootPOA... done
2014-01-25 08:07:10,651 err: 2014-01-25 08:07:10.651 FINE POA RootPOAstop the request controller ...
2014-01-25 08:07:10,651 err: 2014-01-25 08:07:10.651 FINE POA RootPOA... done
2014-01-25 08:07:10,651 err: 2014-01-25 08:07:10.651 INFO POA RootPOA destroyed
2014-01-25 08:07:10,652 err: 2014-01-25 08:07:10.652 INFO prepare ORB for shutdown...
2014-01-25 08:07:10,652 err: 2014-01-25 08:07:10.652 INFO ORB going down...
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.652 FINE [0/0] shutting down pool
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.652 FINE GIOPConnectionManager.shutdown(), 0 connections
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.652 FINE ClientGIOPConnection to 172.17.131.19:41867 (752462d5): close()
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.652 INFO Client-side TCP transport to 172.17.131.19:41867 closed.
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.652 FINE ClientConnectionManager shut down (all connections released)
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.653 FINE [0/1] shutting down pool
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.653 INFO ORB shutdown complete
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.654 FINE Caught acceptor event: org.jacorb.orb.listener.AcceptorExceptionEvent[source=Thread[ServerSocketListener,5,main]] with ORB org.jacorb.orb.ORB@6e0d2a4e and trace java.net.SocketException: Socket closed
2014-01-25 08:07:10,654 err: 2014-01-25 08:07:10.654 INFO Listener exited
2014-01-25 08:07:10,656 err: 2014-01-25 08:07:10.654 FINE Transport to 172.17.131.19:41867: stream closed Socket closed
2014-01-25 08:07:10,657 err: 2014-01-25 08:07:10.656 FINE Unknown exception type java.net.SocketException with exception java.net.SocketException: Socket closed
2014-01-25 08:07:10,657 err: 2014-01-25 08:07:10.657 FINE Caught exception
2014-01-25 08:07:10,657 err:     java.net.SocketInputStream.socketRead0(Native Method)
2014-01-25 08:07:10,657 err:     java.net.SocketInputStream.read(SocketInputStream.java:152)
2014-01-25 08:07:10,657 err:     java.net.SocketInputStream.read(SocketInputStream.java:122)
2014-01-25 08:07:10,657 err:     org.jacorb.orb.etf.StreamConnectionBase.read(Unknown Source)
2014-01-25 08:07:10,657 err:     org.jacorb.orb.giop.GIOPConnection.getMessage(Unknown Source)
2014-01-25 08:07:10,657 err:     org.jacorb.orb.giop.GIOPConnection.receiveMessages(Unknown Source)
2014-01-25 08:07:10,657 err:     org.jacorb.orb.giop.MessageReceptor.doWork(Unknown Source)
2014-01-25 08:07:10,657 err:     org.jacorb.util.threadpool.ConsumerTie.run(Unknown Source)
2014-01-25 08:07:10,657 err:     java.lang.Thread.run(Thread.java:744)
2014-01-25 08:07:10,657 err: 2014-01-25 08:07:10.657 FINE ClientGIOPConnection to 172.17.131.19:41867 (752462d5): getMessage() -- COMM_FAILURE
2014-01-25 08:07:10,657 err: 2014-01-25 08:07:10.657 FINE ClientGIOPConnection to 172.17.131.19:41867 (752462d5): streamClosed()
2014-01-25 08:07:10,657 err: 2014-01-25 08:07:10.657 FINE ClientGIOPConnection to 172.17.131.19:41867 (752462d5): closeAllowReopen()
2014-01-25 08:07:10,657 err: 2014-01-25 08:07:10.657 INFO ConsumerTie exited