-
Bug
-
Resolution: Done
-
Major
-
None
-
None
Sometimes indexing will throw an error such as
at org.infinispan.search.mapper.work.impl.SearchIndexerImpl.lambda$start$0(SearchIndexerImpl.java:68) at org.infinispan.search.mapper.work.impl.SearchIndexerImpl$$Lambda/0x00007deb6f8b4420.accept(Unknown Source:-1) at io.reactivex.rxjava3.internal.operators.flowable.FlowableOnBackpressureLatest$BackpressureLatestSubscriber.onNext(FlowableOnBackpressureLatest.java:52) at io.reactivex.rxjava3.processors.UnicastProcessor.drainRegular(UnicastProcessor.java:309) at io.reactivex.rxjava3.processors.UnicastProcessor.drain(UnicastProcessor.java:384) at io.reactivex.rxjava3.processors.UnicastProcessor.onNext(UnicastProcessor.java:444) at io.reactivex.rxjava3.internal.util.NotificationLite.acceptFull(NotificationLite.java:273) at io.reactivex.rxjava3.internal.util.AppendOnlyLinkedArrayList.accept(AppendOnlyLinkedArrayList.java:119) at io.reactivex.rxjava3.processors.SerializedProcessor.emitLoop(SerializedProcessor.java:176) at io.reactivex.rxjava3.processors.SerializedProcessor.onNext(SerializedProcessor.java:104) at org.infinispan.search.mapper.work.impl.SearchIndexerImpl.sendOperation(SearchIndexerImpl.java:112) at org.infinispan.search.mapper.work.impl.SearchIndexerImpl.lambda$purge$10(SearchIndexerImpl.java:161) at org.infinispan.search.mapper.work.impl.SearchIndexerImpl$$Lambda/0x00007deb6f915408.apply(Unknown Source:-1) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:636) at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291) at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:656) at org.infinispan.search.mapper.work.impl.SearchIndexerImpl.purge(SearchIndexerImpl.java:165) at org.infinispan.query.backend.QueryInterceptor.removeFromIndexes(QueryInterceptor.java:384) at org.infinispan.query.backend.QueryInterceptor.processChange(QueryInterceptor.java:439) at org.infinispan.query.backend.QueryInterceptor.indexIfNeeded(QueryInterceptor.java:248) at org.infinispan.query.backend.QueryInterceptor.lambda$handleManyWriteCommand$2(QueryInterceptor.java:215) at org.infinispan.query.backend.QueryInterceptor$$Lambda/0x00007deb6f7e5ad0.apply(Unknown Source:-1) at org.infinispan.commons.util.concurrent.CompletionStages.lambda$performConcurrently$3(CompletionStages.java:320) at org.infinispan.commons.util.concurrent.CompletionStages$$Lambda/0x00007deb6f7e2ef0.apply(Unknown Source:-1) at io.reactivex.rxjava3.internal.operators.flowable.FlowableConcatMap$ConcatMapImmediate.drain(FlowableConcatMap.java:266) at io.reactivex.rxjava3.internal.operators.flowable.FlowableConcatMap$BaseConcatMapSubscriber.innerComplete(FlowableConcatMap.java:171) at io.reactivex.rxjava3.internal.operators.flowable.FlowableConcatMap$ConcatMapInner.onComplete(FlowableConcatMap.java:587) at io.reactivex.rxjava3.processors.AsyncProcessor$AsyncSubscription.onComplete(AsyncProcessor.java:359) at io.reactivex.rxjava3.processors.AsyncProcessor.onComplete(AsyncProcessor.java:197) at org.infinispan.commons.reactive.RxJavaInterop.lambda$voidCompletionStageToFlowable$0(RxJavaInterop.java:91) at org.infinispan.commons.reactive.RxJavaInterop$$Lambda/0x00007deb6f919498.accept(Unknown Source:-1) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2223) at org.infinispan.search.mapper.work.impl.SearchIndexerImpl.lambda$sendOperation$2(SearchIndexerImpl.java:95) at org.infinispan.search.mapper.work.impl.SearchIndexerImpl$$Lambda/0x00007deb6f919260.accept(Unknown Source:-1) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885) at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:526) at io.netty.util.concurrent.AbstractEventExecutor.runTask$$$capture(AbstractEventExecutor.java:173) at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:-1) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:166) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:-1) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
However, when reproducing and debugging the queues are not full and each of the processors in each step have requests, showing there is no sign of back pressure issues.
Further investigating I was able to figure this was related to a specific operator onBackpressureLatest which can present an object as being dropped due to back pressure but is actually due to how the operator was implemented ReactiveX/RxJava#7864. If we use onBackpressureDrop instead it only pays attention to request limits and thus doesn't run into this issue unless there is actually a backpressure issue.
- links to