Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-13357

(Regression) Execution of concurrent batch jobs containg partitioned steps causes deadlock

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Blocker
    • 20.0.0.Final
    • 19.0.0.Final
    • Batch
    • None

    Description

      Hello,

      the issue described in JBERET-180 seems to have reappeared. I am running Wildfly 16 with jberet-1.3.3. Given that there is a default batch-thread count of 10 I was able to produce a deadlock by starting 10 instances of a partitioned job simultaneously. None of the job runs fast enough to finish before all 10 jobs have been started. All 10 Batch-threads are stuck here:

      "Batch Thread - 1@33537" prio=5 tid=0x109 nid=NA waiting
  java.lang.Thread.State: WAITING
	  at jdk.internal.misc.Unsafe.park(Unknown Source:-1)
	  at java.util.concurrent.locks.LockSupport.park(Unknown Source:-1)
	  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source:-1)
	  at java.util.concurrent.ArrayBlockingQueue.take(Unknown Source:-1)
	  at org.jberet.runtime.runner.StepExecutionRunner.beginPartition(StepExecutionRunner.java:350)
	  at org.jberet.runtime.runner.StepExecutionRunner.runBatchletOrChunk(StepExecutionRunner.java:222)
	  at org.jberet.runtime.runner.StepExecutionRunner.run(StepExecutionRunner.java:144)
	  at org.jberet.runtime.runner.CompositeExecutionRunner.runStep(CompositeExecutionRunner.java:164)
	  at org.jberet.runtime.runner.CompositeExecutionRunner.runFromHeadOrRestartPoint(CompositeExecutionRunner.java:88)
	  at org.jberet.runtime.runner.JobExecutionRunner.run(JobExecutionRunner.java:60)
	  at org.wildfly.extension.batch.jberet.deployment.BatchEnvironmentService$WildFlyBatchEnvironment$1.run(BatchEnvironmentService.java:180)
	  at org.wildfly.extension.requestcontroller.RequestController$QueuedTask$1.run(RequestController.java:494)
	  at org.jberet.spi.JobExecutor$2.run(JobExecutor.java:149)
	  at org.jberet.spi.JobExecutor$1.run(JobExecutor.java:99)
	  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source:-1)
	  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source:-1)
	  at java.lang.Thread.run(Unknown Source:-1)
	  at org.jboss.threads.JBossThread.run(JBossThread.java:485)

      which is this line of code:

      completedPartitionThreads.take();

      Rarely some threads also get stuck at line 364 instead, which is

      final Serializable data = collectorDataQueue.take();

      Attachments

        Issue Links

          is blocked by

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFLY-13428 Upgrade to WildFly Core 12.0.0.Beta3

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) WFCORE-4949 Expose the original task in RequestController.QueuedTask

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFLY-13416 Upgrade jberet-core from 1.3.5.Final to 1.3.6.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Bug - A problem that impairs or prevents the functions of the product. JBERET-125 Split may cause deadlock

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) JBERET-493 org.jberet.spi.JobExecutor#wrap method should allow override by subclass

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Activity

            People

              cfang@redhat.com Cheng Fang
              felk_ Felix König (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: