Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-3604

Enabling Pooling for the LDAP Login Module Causes Shutdown to Hang

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Explained
    • AMQ 7.3.0.GA, AMQ 7.4.0.GA, AMQ 7.2.3.GA, AMQ 7.5.0.GA, AMQ 7.6.0.GA
    • None
    • broker-core, security
    • None
    • Hide

      1. Configure the broker with an LDAP login module with connection pooling enabled:

      activemq {
      
        org.apache.activemq.artemis.spi.core.security.jaas.LDAPLoginModule required
              debug=true
              initialContextFactory=com.sun.jndi.ldap.LdapCtxFactory
              connectionURL="ldap://ldap.host.example.redhat.com:389"
              connectionUsername="uid=admin,cn=users,cn=accounts,dc=redhat,dc=example"
              connectionPassword="XXXXXX"
              connectionProtocol=simple
              authentication=simple
              userBase="dc=redhat,dc=example"
              userSearchMatching="(uid={0})"
              userSearchSubtree=true
              roleBase="dc=redhat,dc=example"
              roleName=cn
              roleSearchMatching="(memberUid={1})"
              roleSearchSubtree=true
              connectionPool=true;
      
      };
      

      2. Add JVM arguments to artemis.profile to bound the pool and set a pool timeout for connections:

      JAVA_ARGS=" -XX:+PrintClassHistogram -XX:+UseG1GC -Xms512M -Xmx2G -Dhawtio.realm=activemq  -Dhawtio.offline=true -Dhawtio.role=amq -Dhawtio.rolePrincipalClasses=org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal -Djolokia.policyLocation=${ARTEMIS_INSTANCE_ETC_URI}jolokia-access.xml -Djon.id=amq -Djon.id=amq -Dcom.sun.jndi.ldap.connect.pool.maxsize=10 -Dcom.sun.jndi.ldap.connect.pool.timeout=30000"
      

      3. Start the broker interactively with ./artemis run

      4. Fire up some consumers to fill the pool until netstat -anp | grep <ldap host>:<ldap port> shows 10 ESTABLISHED connections

      5. Stop consumers

      6. Wait for some time and re-run netstat to observe connections remain open to ldap provider

      7. Stop the broker with Ctrl-C

      8. Broker hangs and does not shutdown (I waited overnight and broker was still alive)

      Show
      1. Configure the broker with an LDAP login module with connection pooling enabled: activemq { org.apache.activemq.artemis.spi.core.security.jaas.LDAPLoginModule required debug= true initialContextFactory=com.sun.jndi.ldap.LdapCtxFactory connectionURL= "ldap: //ldap.host.example.redhat.com:389" connectionUsername= "uid=admin,cn=users,cn=accounts,dc=redhat,dc=example" connectionPassword= "XXXXXX" connectionProtocol=simple authentication=simple userBase= "dc=redhat,dc=example" userSearchMatching= "(uid={0})" userSearchSubtree= true roleBase= "dc=redhat,dc=example" roleName=cn roleSearchMatching= "(memberUid={1})" roleSearchSubtree= true connectionPool= true ; }; 2. Add JVM arguments to artemis.profile to bound the pool and set a pool timeout for connections: JAVA_ARGS= " -XX:+PrintClassHistogram -XX:+UseG1GC -Xms512M -Xmx2G -Dhawtio.realm=activemq -Dhawtio.offline= true -Dhawtio.role=amq -Dhawtio.rolePrincipalClasses=org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal -Djolokia.policyLocation=${ARTEMIS_INSTANCE_ETC_URI}jolokia-access.xml -Djon.id=amq -Djon.id=amq -Dcom.sun.jndi.ldap.connect.pool.maxsize=10 -Dcom.sun.jndi.ldap.connect.pool.timeout=30000" 3. Start the broker interactively with ./artemis run 4. Fire up some consumers to fill the pool until netstat -anp | grep <ldap host>:<ldap port> shows 10 ESTABLISHED connections 5. Stop consumers 6. Wait for some time and re-run netstat to observe connections remain open to ldap provider 7. Stop the broker with Ctrl-C 8. Broker hangs and does not shutdown (I waited overnight and broker was still alive)
    • Hide
      If you enable connection pooling for an LDAP provider (that is, by setting `connectionPool` to `true` in the `LDAPLoginModule` section of the `login.config` configuration file), this can cause connections to the LDAP provider to remain open indefinitely, even when you stop the broker clients. As a result, if you try to shut down the broker in the normal way, the broker does not shut down. Instead, you need to use a Linux command such as `SIGKILL` to terminate the broker process. This situation occurs even if you specify a pool timeout in the JVM arguments for the broker (for example, `-Dcom.sun.jndi.ldap.connect.pool.timeout=30000`) and there are no active clients when you try to shut down the broker.

      To work around this issue, set a value for the `connectionTimeout` property in the `LDAPLoginModule` section of the `login.config` configuration file. When connection pooling has been requested for a connection, the `connectionTimeout` property specifies the maximum time that the broker waits for a connection when the maximum pool size has already been reached and all connections in the pool are in use.
      Show
      If you enable connection pooling for an LDAP provider (that is, by setting `connectionPool` to `true` in the `LDAPLoginModule` section of the `login.config` configuration file), this can cause connections to the LDAP provider to remain open indefinitely, even when you stop the broker clients. As a result, if you try to shut down the broker in the normal way, the broker does not shut down. Instead, you need to use a Linux command such as `SIGKILL` to terminate the broker process. This situation occurs even if you specify a pool timeout in the JVM arguments for the broker (for example, `-Dcom.sun.jndi.ldap.connect.pool.timeout=30000`) and there are no active clients when you try to shut down the broker. To work around this issue, set a value for the `connectionTimeout` property in the `LDAPLoginModule` section of the `login.config` configuration file. When connection pooling has been requested for a connection, the `connectionTimeout` property specifies the maximum time that the broker waits for a connection when the maximum pool size has already been reached and all connections in the pool are in use.
    • Not Required

    Description

      Enabling pooling of connections to the LDAP provider in the login.config causes connections to the LDAP provider to hang indefinitely, forcing the broker to be killed with SIGKILL. This occurs even if a pool timeout is applied to the JVM arguments (-Dcom.sun.jndi.ldap.connect.pool.timeout=30000) and if there are no active clients at the time of shutdown.

      Attachments

        1. artemis.log
          31 kB
        2. high-cpu-tdump.out
          828 kB
        3. tdump.tar.gz
          84 kB

        Issue Links

          Activity

            People

              rhn-support-jbertram Justin Bertram
              rhn-support-dhawkins Duane Hawkins
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: