Environment

• Server: RHDG 8.5.3 on OCP 4, datagrid-operator.v8.5.7
• Client: Hot Rod of RHDG 8.5.3
• RHDG Server and Hot Rod Client run on separated OCP namespace. Hot Rod Client access to RHDG Server via OCP Service of RHDG Server's namespace.

Issue

After upgrading to Red Hat Data Grid (RHDG) 8.5.3, the Hot Rod Client side has started outputting a WARN log: 

<DATE?TIME> WARN 1 --- [async-pool-1-16] org.infinispan.HOTROD                    : ISPN004005: Error received from the server: javax.security.sasl.SaslException: ELY05053: Callback handler failed for unknown reason [Caused by java.util.ConcurrentModificationException] 

The phenomenon began occurring after an upgrade from Data Grid 8.4 to 8.5.3, and it has been confirmed to also occur in a separate environment.
 
Regarding reproducibility, it was reported to occur 2 out of 2 times in a specific environment.
 
Logs from the RHDG server side show `SASL Negotiation Failed` and `Handling AuthenticationCompleteCallback: fail` recorded on the `blocking-thread--p3-t13` thread when an authentication request is received from the Hot Rod client.

<Date/TIME> TRACE (blocking-thread--p3-t13) [org.wildfly.security.sasl.digest] SASL Negotiation Failed
<Date/TIME> TRACE (blocking-thread--p3-t13) [org.wildfly.security] Handling AuthenticationCompleteCallback: fail 

The `ISPN004005` WARN log and the accompanying `java.util.ConcurrentModificationException` are believed to be caused by two threads executing authentication-related processes almost simultaneously on the RHDG server side when responding to an authentication request from the Hot Rod client. (I will provide a separate comment on this point.)