Uploaded image for project: 'Drools'
  1. Drools
  2. DROOLS-7542

ArrayIndexOutOfBoundsException during parallel building of rules with shared parents

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • None
    • 8.43.0.Final
    • None
    • None
    • 2023 Week 33-35 (from Aug 14)
    • Hide

      I used the following to build a container.

      KieServices ks = KieServices.Factory.get();
KieRepository kr = ks.getRepository();
KieFileSystem kfs = ks.newKieFileSystem();
// Add DRL rules here with: kfs.write(resource);
KieBuilder kb = ks.newKieBuilder(kfs);
kb.buildAll();
kc = ks.newKieContainer(kr.getDefaultReleaseId());

      I can't share my rules (because I don't know the project policy on that), but most are referencing a parent rule with "extends". I have 88 rules in total, and the exception is thrown in about 1 out of 30 times during initialization.

      Show
      I used the following to build a container. KieServices ks = KieServices.Factory.get(); KieRepository kr = ks.getRepository(); KieFileSystem kfs = ks.newKieFileSystem(); // Add DRL rules here with: kfs.write(resource); KieBuilder kb = ks.newKieBuilder(kfs); kb.buildAll(); kc = ks.newKieContainer(kr.getDefaultReleaseId()); I can't share my rules (because I don't know the project policy on that), but most are referencing a parent rule with "extends". I have 88 rules in total, and the exception is thrown in about 1 out of 30 times during initialization.
    • NEW
    • NEW
    • ---
    • ---

    Description

      Often during building of rules that extend other rules, an ArrayIndexOutOfBoundsException is thrown:

      java.lang.RuntimeException: Rules compilation failed or interrupted
    at org.drools.compiler.builder.impl.processors.ImmutableRuleCompilationPhase.compileRulesLevel(ImmutableRuleCompilationPhase.java:285)
    at org.drools.compiler.builder.impl.processors.ImmutableRuleCompilationPhase.process(ImmutableRuleCompilationPhase.java:82)
    at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4204)
    at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:461)
    at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildRules(KnowledgeBuilderImpl.java:1020)
    at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.doFirstBuildStep(KnowledgeBuilderImpl.java:994)
    at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:114)
    at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:104)
    at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:270)
    at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:216)
    at org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:80)
    at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:282)
    at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:248)
    at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:195)
    at com.example.TestCompilation(TestCompilation.java:32)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException
    at java.base/java.util.concurrent.ForkJoinTask.reportExecutionException(ForkJoinTask.java:605)
    at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:981)
    at org.drools.compiler.builder.impl.processors.ImmutableRuleCompilationPhase.compileRulesLevel(ImmutableRuleCompilationPhase.java:283)
    ... 16 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
    at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:564)
    at java.base/java.util.concurrent.ForkJoinTask.reportExecutionException(ForkJoinTask.java:604)
    ... 18 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
    at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:564)
    at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
    at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:689)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
    at org.drools.compiler.builder.impl.processors.ImmutableRuleCompilationPhase.lambda$compileRulesLevel$4(ImmutableRuleCompilationPhase.java:272)
    at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1375)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0
    at java.base/java.util.ArrayList.add(ArrayList.java:455)
    at java.base/java.util.ArrayList.add(ArrayList.java:467)
    at org.drools.base.definitions.rule.impl.RuleImpl.addChild(RuleImpl.java:813)
    at org.drools.base.definitions.rule.impl.RuleImpl.setParent(RuleImpl.java:802)
    at org.drools.compiler.rule.builder.RuleBuilder.preProcess(RuleBuilder.java:77)
    at org.drools.compiler.builder.impl.processors.ImmutableRuleCompilationPhase.buildRuleBuilderContext(ImmutableRuleCompilationPhase.java:260)
    at org.drools.compiler.builder.impl.processors.ImmutableRuleCompilationPhase.lambda$compileRulesLevel$3(ImmutableRuleCompilationPhase.java:274)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
    at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
    ... 5 more

      It appears to happen because rules are by default built in parallel, yet the parent-child relationship in a rule class is not thread-safe. From ImmutableRuleCompilationPhase:

      private void compileRulesLevel(PackageDescr packageDescr, PackageRegistry pkgRegistry, List<RuleDescr> rules) {
    boolean parallelRulesBuild = parallelRulesBuild(rules);
    if (parallelRulesBuild) {
        Map<String, RuleBuildContext> ruleCxts = new ConcurrentHashMap<>();
        try {
            KnowledgeBuilderImpl.ForkJoinPoolHolder.COMPILER_POOL.submit(() ->
                    rules.stream().parallel() // <-- Rules are built in parallel
                            .filter(ruleDescr -> filterAccepts(ResourceChange.Type.RULE, ruleDescr.getNamespace(), ruleDescr.getName()))
                            .forEach(ruleDescr -> {
                                initRuleDescr(packageDescr, pkgRegistry, ruleDescr);
                                RuleBuildContext context = buildRuleBuilderContext(pkgRegistry, ruleDescr); // <-- Exception happens from this place in the call stack
                                ruleCxts.put(ruleDescr.getName(), context);
                                List<? extends KnowledgeBuilderResult> results = addRule(context);
                                if (!results.isEmpty()) {
                                    synchronized (this.results) {
                                        this.results.addAll(results);
                                    }
                                }
                            })
            ).get();
        //...

      For my rules, many that extends the same parent rule, the above eventually calls addChild on a shared parent RuleImpl:

      public void addChild(RuleImpl child) {
    if (children == null) {
        children = new ArrayList<>();
    }
    children.add(child);
}

      The above modification to a list is not threadsafe, though it is accessed concurrently. A solution might be to change the type from ArrayList to one that allows concurrent access or to synchronize accessors.

      As a workaround, I have disabled parallel rule building with:

      System.getProperties().put(ParallelRulesBuildThresholdOption.PROPERTY_NAME, "-1");

      Attachments

        Activity

          People

            mfusco@redhat.com Mario Fusco
            bmv@netcompany.com Bjørn Vester (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: