[ISPN-12253] No Marshaller registered for RebalancingStatus enum class alongside Prometheus

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: None
Affects Version/s: 10.1.8.Final
Component/s: None
Labels:
None

Steps to Reproduce:

Hide

1. Initiate infinispan cache manager through a blueprint
2. Include the bundle to start up with the karaf features
3. Run application in a docker container

Show
1. Initiate infinispan cache manager through a blueprint 2. Include the bundle to start up with the karaf features 3. Run application in a docker container
Workaround Description:
Hide

<serialization marshaller="org.infinispan.commons.marshall.JavaSerializationMarshaller"> <white-list> <class>org.infinispan.topology.RebalancingStatus</class> </white-list> </serialization>
Show
<serialization marshaller= "org.infinispan.commons.marshall.JavaSerializationMarshaller" > <white-list> < class& gt;org.infinispan.topology.RebalancingStatus</ class& gt; </white-list> </serialization>

Hello,

We encountered a very strange issue running infinispan with two instances of our application that connect to the same cluster. Once Prometheus is ran along with the other instances, it caused the following error.

Versions:
Karaf: 4.2.6
Infinispan: 10.1.8.Final
Docker: 19.03.12, build 48a66213fe

java.lang.IllegalArgumentException: No marshaller registered for Java type org.infinispan.topology.RebalancingStatus    at org.infinispan.protostream.impl.SerializationContextImpl.getMarshallerDelegate(SerializationContextImpl.java:279) ~[422:org.infinispan.protostream:4.3.2.Final]    at org.infinispan.protostream.WrappedMessage.writeMessage(WrappedMessage.java:240) ~[422:org.infinispan.protostream:4.3.2.Final]    at org.infinispan.protostream.ProtobufUtil.toWrappedStream(ProtobufUtil.java:196) ~[422:org.infinispan.protostream:4.3.2.Final]    at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToBuffer(PersistenceMarshallerImpl.java:157) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToByteBuffer(PersistenceMarshallerImpl.java:137) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToByteBuffer(PersistenceMarshallerImpl.java:145) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeRawUnknown(GlobalMarshaller.java:638) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeUnknown(GlobalMarshaller.java:627) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeUnknown(GlobalMarshaller.java:618) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:384) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:352) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.BytesObjectOutput.writeObject(BytesObjectOutput.java:26) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.writeObject(SuccessfulResponse.java:65) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.writeObject(SuccessfulResponse.java:62) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeInternal(GlobalMarshaller.java:656) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:371) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:352) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:181) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:174) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.marshall.core.GlobalMarshaller.objectToBuffer(GlobalMarshaller.java:302) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.sendResponse(JGroupsTransport.java:1355) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$processRequest$8(JGroupsTransport.java:1400) ~[421:org.infinispan.core:10.1.5.Final]    at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ResponseConsumer.accept(GlobalInboundInvocationHandler.java:182) [421:org.infinispan.core:10.1.5.Final]    at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ResponseConsumer.accept(GlobalInboundInvocationHandler.java:170) [421:org.infinispan.core:10.1.5.Final]    at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) [?:?]    at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) [?:?]    at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) [?:?]    at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ReplicableCommandRunner.run(GlobalInboundInvocationHandler.java:160) [421:org.infinispan.core:10.1.5.Final]    at org.infinispan.util.concurrent.BlockingTaskAwareExecutorServiceImpl$RunnableWrapper.run(BlockingTaskAwareExecutorServiceImpl.java:215) [421:org.infinispan.core:10.1.5.Final]    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]    at java.lang.Thread.run(Thread.java:748) [?:?]

Thanks in advance!

Pedro Zapata Fernandez made changes - 2023/11/16 2:32 PM

Workflow

Original: GIT Pull Request with Triage workflow [ 13280237 ]

New: OJA-WF-BG [ 24695968 ]

Roni Polus (Inactive) made changes - 2020/08/28 5:37 PM

Resolution		New: Done [ 1 ]
Status	Original: New [ 10016 ]	New: Closed [ 6 ]

Roni Polus (Inactive) made changes - 2020/08/28 5:37 PM

Description

Original: Hello,

We encountered a very strange issue running infinispan inside OSGi. We have a cache setup and when running locally (without a container), karaf does not report any problems with the cache and it runs just fine. Although, when running inside a container, we get the following error spammed through the karaf logs:

Versions:
Karaf: 4.2.6
Infinispan: 10.1.8.Final
Docker: 19.03.12, build 48a66213fe

{code:java}
java.lang.IllegalArgumentException: No marshaller registered for Java type org.infinispan.topology.RebalancingStatus at org.infinispan.protostream.impl.SerializationContextImpl.getMarshallerDelegate(SerializationContextImpl.java:279) ~[422:org.infinispan.protostream:4.3.2.Final] at org.infinispan.protostream.WrappedMessage.writeMessage(WrappedMessage.java:240) ~[422:org.infinispan.protostream:4.3.2.Final] at org.infinispan.protostream.ProtobufUtil.toWrappedStream(ProtobufUtil.java:196) ~[422:org.infinispan.protostream:4.3.2.Final] at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToBuffer(PersistenceMarshallerImpl.java:157) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToByteBuffer(PersistenceMarshallerImpl.java:137) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToByteBuffer(PersistenceMarshallerImpl.java:145) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeRawUnknown(GlobalMarshaller.java:638) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeUnknown(GlobalMarshaller.java:627) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeUnknown(GlobalMarshaller.java:618) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:384) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:352) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.BytesObjectOutput.writeObject(BytesObjectOutput.java:26) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.writeObject(SuccessfulResponse.java:65) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.writeObject(SuccessfulResponse.java:62) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeInternal(GlobalMarshaller.java:656) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:371) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:352) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:181) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:174) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.objectToBuffer(GlobalMarshaller.java:302) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.sendResponse(JGroupsTransport.java:1355) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$processRequest$8(JGroupsTransport.java:1400) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ResponseConsumer.accept(GlobalInboundInvocationHandler.java:182) [421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ResponseConsumer.accept(GlobalInboundInvocationHandler.java:170) [421:org.infinispan.core:10.1.5.Final] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) [?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) [?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) [?:?] at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ReplicableCommandRunner.run(GlobalInboundInvocationHandler.java:160) [421:org.infinispan.core:10.1.5.Final] at org.infinispan.util.concurrent.BlockingTaskAwareExecutorServiceImpl$RunnableWrapper.run(BlockingTaskAwareExecutorServiceImpl.java:215) [421:org.infinispan.core:10.1.5.Final] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?] at java.lang.Thread.run(Thread.java:748) [?:?]
{code}
I debugged through the stacktrace while running locally and it seems like `RebalancingStatus` is never passed to `getMarshallerDelegate`

We are using bluePrint to reference the cache manager
{code:java}
<reference id="cacheManager" interface="org.infinispan.manager.EmbeddedCacheManager"/>

<service interface="our.class.Cache">
    <bean class="our.class.CacheImpl">
        <argument ref="cacheManager"/>
    </bean>
</service>
{code}
I can provide more context upon request.

Thanks in advance!

New: Hello,

We encountered a very strange issue running infinispan with two instances of our application that connect to the same cluster. Once Prometheus is ran along with the other instances, it caused the following error.

Versions:
Karaf: 4.2.6
Infinispan: 10.1.8.Final
Docker: 19.03.12, build 48a66213fe
{code:java}
java.lang.IllegalArgumentException: No marshaller registered for Java type org.infinispan.topology.RebalancingStatus at org.infinispan.protostream.impl.SerializationContextImpl.getMarshallerDelegate(SerializationContextImpl.java:279) ~[422:org.infinispan.protostream:4.3.2.Final] at org.infinispan.protostream.WrappedMessage.writeMessage(WrappedMessage.java:240) ~[422:org.infinispan.protostream:4.3.2.Final] at org.infinispan.protostream.ProtobufUtil.toWrappedStream(ProtobufUtil.java:196) ~[422:org.infinispan.protostream:4.3.2.Final] at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToBuffer(PersistenceMarshallerImpl.java:157) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToByteBuffer(PersistenceMarshallerImpl.java:137) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectToByteBuffer(PersistenceMarshallerImpl.java:145) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeRawUnknown(GlobalMarshaller.java:638) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeUnknown(GlobalMarshaller.java:627) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeUnknown(GlobalMarshaller.java:618) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:384) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:352) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.BytesObjectOutput.writeObject(BytesObjectOutput.java:26) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.writeObject(SuccessfulResponse.java:65) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.writeObject(SuccessfulResponse.java:62) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeInternal(GlobalMarshaller.java:656) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:371) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:352) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:181) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:174) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.marshall.core.GlobalMarshaller.objectToBuffer(GlobalMarshaller.java:302) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.sendResponse(JGroupsTransport.java:1355) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$processRequest$8(JGroupsTransport.java:1400) ~[421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ResponseConsumer.accept(GlobalInboundInvocationHandler.java:182) [421:org.infinispan.core:10.1.5.Final] at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ResponseConsumer.accept(GlobalInboundInvocationHandler.java:170) [421:org.infinispan.core:10.1.5.Final] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) [?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) [?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) [?:?] at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ReplicableCommandRunner.run(GlobalInboundInvocationHandler.java:160) [421:org.infinispan.core:10.1.5.Final] at org.infinispan.util.concurrent.BlockingTaskAwareExecutorServiceImpl$RunnableWrapper.run(BlockingTaskAwareExecutorServiceImpl.java:215) [421:org.infinispan.core:10.1.5.Final] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?] at java.lang.Thread.run(Thread.java:748) [?:?]
{code}

Thanks in advance!

Roni Polus (Inactive) added a comment - 2020/08/28 5:35 PM

After a deeper research, we figured out that having Prometheus container running along with two instances of our application that connect to one infinispan cluster up at the same time resolved in the error spamming in one of the instances.

To resolve, we had to remove infinispan object being exported via `jmx_prometheus_javaagent`

Roni Polus (Inactive) added a comment - 2020/08/28 5:35 PM After a deeper research, we figured out that having Prometheus container running along with two instances of our application that connect to one infinispan cluster up at the same time resolved in the error spamming in one of the instances. To resolve, we had to remove infinispan object being exported via `jmx_prometheus_javaagent`

Roni Polus (Inactive) made changes - 2020/08/28 1:31 PM

Summary

Original: No Marshaller registered for RebalancingStatus enum class - Only inside docker container

New: No Marshaller registered for RebalancingStatus enum class alongside Prometheus

Roni Polus (Inactive) created issue - 2020/08/24 7:26 PM

Assignee:: Unassigned

Reporter:: Roni Polus (Inactive)

Archiver:: Amol Dongare

Created:: 2020/08/24 7:26 PM

Updated:: 2020/08/28 5:37 PM

Resolved:: 2020/08/28 5:37 PM

Archived:: 2024/11/28 6:21 AM

Details

Description

Attachments

Easy Agile Planning Poker

Activity

Collapse comment: Roni Polus (Inactive) added a comment - 2020/08/28 5:35 PM

Expand comment: Roni Polus (Inactive) added a comment - 2020/08/28 5:35 PM

People

Dates

PagerDuty