Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-18683

Blocked JDBC store threads prevent shutdown

XMLWordPrintable

      Scenario: we have a 4 nodes cluster using an ORACLE RAC 19 database to offload session data;
      Each node is configured as in the following:

      embed-server --server-config=standalone-ha.xml
      /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp)
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload:add()
      data-source add --name=testDS --jndi-name=java:jboss/datasources/testDS --driver-name=oracle-connector.jar --connection-url=jdbc:oracle:thin:@(DESCRIPTION=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=oracle-19c-rac-01.mwqe.upshift.rdu2.redhat.com)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=oracle-19c-rac-02.mwqe.upshift.rdu2.redhat.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=dballo))) --enabled=true --jta=true --use-java-context=true --transaction-isolation=TRANSACTION_READ_COMMITTED --min-pool-size=1 --max-pool-size=5 --pool-prefill=true --user-name=dballo03 --password=dballo03 --prepared-statements-cache-size=32 --share-prepared-statements=true
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc:add(data-source=testDS,fetch-state=false,passivation=false,purge=false,shared=true,dialect=ORACLE){allow-resource-service-restart=true}
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc/table=string:write-attribute(name=id-column.name,value=id)
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc/table=string:write-attribute(name=data-column.name,value=datum)
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc/table=string:write-attribute(name=timestamp-column.name,value=version)
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc/table=string:write-attribute(name=id-column.type,value=VARCHAR2(255))
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc/table=string:write-attribute(name=timestamp-column.type,value=NUMBER)
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc/table=string:write-attribute(name=data-column.type,value=BLOB)
      /subsystem=infinispan/cache-container=web/invalidation-cache=offload/store=jdbc/table=string:write-attribute(name=prefix, value=i){allow-resource-service-restart=true}
      /subsystem=infinispan/cache-container=web:write-attribute(name=default-cache, value=offload)
      /subsystem=transactions:write-attribute(name=node-identifier,value=wildfly1)
      

      Each node is failed in a sequence by being shut down and subsequently restarted;
      At the end of the test, all node are shut down in the following order node1, node2,node3,node4;

      The issue is that node2 did not shut down after receiving the shut down command;

      Complete logs in [^RUN-119.zip];

      Thread dump of the node that hung in [^stack.log];

      The issue is very difficult to reproduce: we just observed it once so far;

            pferraro@redhat.com Paul Ferraro
            pferraro@redhat.com Paul Ferraro
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: