Summary

Off-heap memory configuration for distributed web session caches fails with ISPN000504 error during application deployment in WildFly 37.0.1.Final and 38.0.0.Beta1 when using PROTOSTREAM marshaller.

Environment

• WildFly Version: 37.0.1.Final, 38.0.0.Beta1
• Infinispan Version: 15.2.5.Final / 15.2.6.Final
• JDK Version: OpenJDK 21.0.8.9
• Configuration: standalone-full-ha.xml
• Clustering: Enabled with JGroups

Description

When configuring a distributed cache for web session management with off-heap memory storage, deployment of web applications fails with:

Caused by: org.infinispan.commons.CacheConfigurationException: ISPN000504: 
Size (bytes) based eviction needs either off-heap or a binary compatible 
storage configured in the cache encoding

Error occurs at:

org.wildfly.clustering.infinispan.service.CacheConfigurationServiceInstaller$4.apply
(CacheConfigurationServiceInstaller.java:72)

Key Finding: Issue occurs only for dynamically created application-specific session caches, NOT for statically defined template caches.

Steps to Reproduce

1. Configure Web Cache Container with Off-Heap Memory

<cache-container name="web" default-cache="dist" 
                 modules="org.wildfly.clustering.session.infinispan.embedded" 
                 marshaller="PROTOSTREAM">
    <transport lock-timeout="60000"/>
    <distributed-cache name="dist" owners="2">
        <locking isolation="REPEATABLE_READ"/>
        <transaction mode="BATCH"/>
        <off-heap-memory size="1" size-unit="GiB"/>
        <expiration interval="0" lifespan="28800000" max-idle="1800000"/>
        <file-store passivation="true" purge="false" 
                    path="sessions" relative-to="jboss.server.data.dir"/>
    </distributed-cache>
</cache-container>

2. Deploy Web Application with distributable tag

<web-app>
    <distributable/>
</web-app>

3. Observe Deployment Failure

WFLYCTL0080: Failed services => 
  org.wildfly.clustering.infinispan.cache-configuration.web."app-name.war"

Expected Behavior

• Off-heap memory should work seamlessly with PROTOSTREAM marshaller
• Application-specific session caches should inherit correct encoding configuration
• Deployment should succeed without errors

Actual Behavior

• Deployment fails with ISPN000504 error
• Dynamically created cache does not have correct binary-compatible encoding set
• Off-heap memory is effectively unusable for web session caching

Root Cause Analysis

Problem Location

File: CacheConfigurationServiceInstaller.java (Line 72)

builder.encoding().mediaType(
    builder.memory().storage().canStoreReferences() 
        ? MediaType.APPLICATION_OBJECT 
        : manager.get().getCacheManagerConfiguration()
                      .serialization()
                      .marshaller()
                      .mediaType()
);

Issue Details

When creating application-specific session cache:

1. Template cache (web/dist) is correctly configured with marshaller="PROTOSTREAM"
2. Code calls manager.get().getCacheManagerConfiguration().serialization().marshaller().mediaType()
3. ProtoStreamMarshaller returns null or empty string for mediaType
4. Encoding is not set to binary-compatible format
5. Infinispan validation in MemoryConfigurationBuilder.checkBinaryRequirement() fails

Proposed Fix

Modify CacheConfigurationServiceInstaller.java at line 72:

StorageType storageType = builder.memory().storage();
if (storageType.canStoreReferences()) {
    builder.encoding().mediaType(MediaType.APPLICATION_OBJECT);
} else {
    String mediaType = manager.get()
        .getCacheManagerConfiguration()
        .serialization()
        .marshaller()
        .mediaType();
    
    // FIX: Provide fallback if marshaller doesn't return mediaType
    if (mediaType == null || mediaType.isEmpty()) {
        mediaType = MediaType.APPLICATION_PROTOSTREAM_TYPE;
    }
    
    builder.encoding().mediaType(mediaType);
}

Workaround

Use heap-based memory instead of off-heap:

<heap-memory max-count="10000"/>

Or remove memory configuration and rely on passivation:

<file-store passivation="true" purge="false" 
            path="sessions" relative-to="jboss.server.data.dir"/>

Related Issues

• JDG-7055: Size based eviction documentation does not detail storage requirements

This appears to be the WildFly-specific manifestation of JDG-7055, as ProtoStreamMarshaller integration in CacheConfigurationServiceInstaller does not properly handle mediaType for dynamically created session caches.

Impact

• High: Off-heap memory unusable for web session caching
• Affects distributed session management performance optimization
• Forces users to use less efficient heap-based storage