Previously I thought it happens just on server shutdown...

Worker node with this mod_cluster and Elytron configuration standalone-ha.xml keeps complaining on being unable to agree on a cipher suite with the Undertow balancer, whose Elytron configuration is identical, i.e.

<server-ssl-contexts>
    <server-ssl-context name="serverSSLContext" providers="openssl" need-client-auth="false" key-managers="keyManager"  protocols="TLSv1.2" trust-managers="trustManager"/>
</server-ssl-contexts>
<client-ssl-contexts>
    <client-ssl-context name="clientSSLContext" providers="openssl" key-managers="clientKeyManager" protocols="TLSv1.2" trust-managers="trustManager"/>
</client-ssl-contexts>

see:

15:19:25,492 FINE  [org.wildfly.openssl.OpenSSLEngine] (UndertowEventHandlerAdapter - 1) The version of SSL in use does not support cipher ordering
15:19:25,541 ERROR [org.jboss.mod_cluster.undertow] (UndertowEventHandlerAdapter - 1) null: java.lang.NullPointerException
    at org.wildfly.openssl.CipherSuiteConverter.toJava(CipherSuiteConverter.java:284)
    at org.wildfly.openssl.OpenSSLEngine.toJavaCipherSuite(OpenSSLEngine.java:1021)
    at org.wildfly.openssl.OpenSSlSession.initCipherSuite(OpenSSlSession.java:305)
    at org.wildfly.openssl.OpenSSlSession.initialised(OpenSSlSession.java:296)
    at org.wildfly.openssl.OpenSSLSessionContext.clientSessionCreated(OpenSSLSessionContext.java:114)
    at org.wildfly.openssl.OpenSSLClientSessionContext.storeClientSideSession(OpenSSLClientSessionContext.java:92)
    at org.wildfly.openssl.OpenSSLEngine.handshakeFinished(OpenSSLEngine.java:940)
    at org.wildfly.openssl.OpenSSLEngine.getHandshakeStatus(OpenSSLEngine.java:989)
    at org.wildfly.openssl.OpenSSLEngine.unwrap(OpenSSLEngine.java:606)
    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
    at org.wildfly.openssl.OpenSSLSocket.runHandshake(OpenSSLSocket.java:316)
    at org.wildfly.openssl.OpenSSLSocket.write(OpenSSLSocket.java:462)
    at org.wildfly.openssl.OpenSSLOutputStream.write(OpenSSLOutputStream.java:46)
    at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
    at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
    at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
    at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
    at java.io.BufferedWriter.flush(BufferedWriter.java:254)
    at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.sendRequest(DefaultMCMPHandler.java:526)
    at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.sendRequest(DefaultMCMPHandler.java:605)
    at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:387)
    at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:365)
    at org.jboss.modcluster.ModClusterService.status(ModClusterService.java:454)
    at org.wildfly.mod_cluster.undertow.UndertowEventHandlerAdapter.run(UndertowEventHandlerAdapter.java:169)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:320)

Full log: server.log.
I might or might not have it misconfigured, but what definitely should not happen is that when I try to shut the server down:

INFO  [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal

it crashes JVM (doublefree?):

# JRE version: Java(TM) SE Runtime Environment (8.0_121-b13) (build 1.8.0_121-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libcrypto.so.1.0.2h+0x117a64]  sk_pop_free+0x24
#

Core: hs_err_pid12281.log

Now, with this configuration, it happens pretty much immediately after the worker tries to register with the balancer:

Elytron

<server-ssl-contexts>
    <server-ssl-context name="serverSSLContext" cipher-suite-filter="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" providers="openssl" need-client-auth="false" key-managers="keyManager"  protocols="TLSv1.2" trust-managers="trustManager"/>
</server-ssl-contexts>
<client-ssl-contexts>
    <client-ssl-context name="clientSSLContext" cipher-suite-filter="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" providers="openssl" key-managers="clientKeyManager" protocols="TLSv1.2" trust-managers="trustManager"/>
</client-ssl-contexts>

Undertow

<server name="default-server">
    <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true"/>
    <https-listener name="https" socket-binding="https" enable-http2="true" ssl-context="serverSSLContext"/>
    <host name="default-host" alias="localhost">
        <location name="/" handler="welcome-content"/>
        <access-log/>
        <filter-ref name="server-header"/>
        <filter-ref name="x-powered-by-header"/>
    </host>
</server>

mod_cluster

<subsystem xmlns="urn:jboss:domain:modcluster:3.0">
    <mod-cluster-config advertise-socket="modcluster" connector="https" ssl-context="clientSSLContext">
        <dynamic-load-provider>
            <load-metric type="cpu"/>
        </dynamic-load-provider>
    </mod-cluster-config>
</subsystem>

See new crashes_on_mod_cluster_connection.server.log and new_hs_err_pid13002.log.

I must admit I am somewhat getting into low spirits with this whole client <--H/2-->mod_cluster balancer<--H/2-->mod_cluster worker and "just use Elytron" situation. Would it be possible for you to actually give it a try? Take 1 Wildfly and Undertow mod_cluster balancer, set it up with H/2 via Elytron, take 1 Wildfly node as H/2 mod_cluster worker configured with Elytron, start it...