It is possible to create server ssl context without key manager.

/subsystem=elytron/server-ssl-context=a:add()

Key manager in elytron holds reference to key store.
I can't think of use case where it would be usefull to configure server ssl context without specifying key store.