Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-3506

Creating and deleting addresses over JMX fails at random

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • AMQ 7.7.0.CR1
    • None

      Download http://messaging-qe-repo.usersys.redhat.com:8081/artifactory/cli-java/amqx-2.9.0.redhat-00005.jar, change broker name from 0.0.0.0 to "amq", enable jmx.

      Run the following against Artemis 2.11 broker. There are three errorneous situations that can happen, when given enough time.

      i=b
while true; do
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action add --name tests.${i}.results.success || break
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action add --name tests.${i}.results.failure || break
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action remove --name tests.${i}.results.success || break
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action remove --name tests.${i}.results.failure || break
i+=1
done;

      1. The broker fails to find previously created address and error is shown in server log, JMX client gets reported success

      Note that error message mentions an auto-created queue, but the destination was actually created explicitly.

      2020-04-21 23:09:18,736 ERROR [org.apache.activemq.artemis.core.server] AMQ224065: Failed to remove auto-created queue tests.b1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111.results.failure: ActiveMQAddressDoesNotExistException[errorType=ADDRESS_DOES_NOT_EXIST message=AMQ229203: Address Does Not Exist: tests.b1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111.results.failure]
	at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.removeAddressInfo(ActiveMQServerImpl.java:3303) [artemis-server-2.11.0.jar:2.11.0]
	at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.removeAddressInfo(ActiveMQServerImpl.java:3292) [artemis-server-2.11.0.jar:2.11.0]
	at org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl$AddressQueueReaper.run(PostOfficeImpl.java:1762) [artemis-server-2.11.0.jar:2.11.0]
	at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:306) [artemis-commons-2.11.0.jar:2.11.0]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.11.0.jar:2.11.0]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.11.0.jar:2.11.0]
	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.11.0.jar:2.11.0]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
	at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.11.0.jar:2.11.0]

      2. The broker fails to find previously created address and error is thrown in JMX client

      java.lang.IllegalStateException: AMQ229203: Address Does Not Exist: tests.b111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111.results.failure
        at org.apache.activemq.artemis.core.management.impl.ActiveMQServerControlImpl.deleteAddress(ActiveMQServerControlImpl.java:870)
        at org.apache.activemq.artemis.core.management.impl.ActiveMQServerControlImpl.deleteAddress(ActiveMQServerControlImpl.java:856)
        at jdk.internal.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
        at jdk.internal.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at java.base/sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:260)
        at java.management/com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
        at java.management/com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
        at java.management/com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
        at java.management/com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
        at java.management/com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
        at java.management/javax.management.StandardMBean.invoke(StandardMBean.java:405)
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:809)
        at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1466)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1307)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1399)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:827)
        at java.base/jdk.internal.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:359)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:690)
        at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:562)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:796)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:677)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:391)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:676)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:835)

      3. When multiple instances of the script above are run at the same time, it can happen that error result about one command is delivered to the other command.

      % i=a
while true; do
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action add --name tests.${i}.results.success || break 
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action add --name tests.${i}.results.failure || break 
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action remove --name tests.${i}.results.success || break
java  -jar amqx-2.9.0.redhat-00005.jar  topic --host 127.0.0.1:1099 --username admin --password admin --action remove --name tests.${i}.results.failure || break
i+=1
done;

topic 'tests.a.results.success' created
topic 'tests.a.results.failure' created
Topic 'tests.a.results.success' removed
Topic 'tests.a.results.failure' removed
topic 'tests.a1.results.success' created
topic 'tests.a1.results.failure' created
Topic 'tests.a1.results.success' removed
Topic 'tests.a1.results.failure' removed
topic 'tests.a11.results.success' created
topic 'tests.a11.results.failure' created
Topic 'tests.a11.results.success' removed
Topic 'tests.a11.results.failure' removed
topic 'tests.a111.results.success' created
Unable to add queue: org.apache.activemq.artemis:broker="amq",component=addresses,address="tests.b1.results.failure"
javax.management.InstanceNotFoundException: org.apache.activemq.artemis:broker="amq",component=addresses,address="tests.b1.results.failure"
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083)
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:637)
        at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1443)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1307)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1399)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:637)
        at java.base/jdk.internal.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:359)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:690)
        at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:562)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:796)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:677)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:391)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:676)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:835)
        at java.rmi/sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283)
        at java.rmi/sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260)
        at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
        at jdk.remoteref/jdk.jmx.remote.internal.rmi.PRef.invoke(Unknown Source)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
        at java.management.rmi/javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:904)
        at java.management/javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:273)
        at com.sun.proxy.$Proxy4.getQueueNames(Unknown Source)
        at com.redhat.amqx.management.artemis.AbstractArtemisManager.getTopics(AbstractArtemisManager.java:77)
        at com.redhat.amqx.management.artemis.TopicArtemisManager.destinationExists(TopicArtemisManager.java:29)
        at com.redhat.amqx.management.artemis.TopicArtemisManager.addDestination(TopicArtemisManager.java:99)
        at com.redhat.amqx.main.actions.TopicAction.run(TopicAction.java:95)
        at com.redhat.amqx.main.Main.main(Main.java:60)

      Note that this command is creating an a* addresses, but the error message talks about a b* address.

              dbruscin Domenico Francesco Bruscino
              jdanek@redhat.com Jiri Daněk
              Tiago Bueno Tiago Bueno
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: