In our setup of 4 Infinispan nodes, one of the nodes at some point got into very high CPU usage state and eventually left the cluster. After looking at the logs, we found an error message saying 
There was a problem moving indexes for compactor with file 72 java.lang.IllegalStateException: Too many records for this key (short overflow).
 
According to wburns@redhat.com (discussing in Zulipchat) this may happen when there are too many records for the same key. Which may be the issue in our case (but we are not sure) because we use Keycloak as the Hotrod client for Infinispan, and Keycloak will keep sessions in Infinispan for as long as we want, and just refresh it when user requests a new access token. As suggested by William, I will try reducing the max file size parameter for SIFS, but this is only short term solution for now (will see if it prevents from future issues).
 
This error only happened on one of the nodes, but across several files during Compaction. Attached the Profiling screenshots and Stack traces from logs.