SSL_KEY_EXCHANGE: Support reloading keystore from disk and allow user to provide explicit truststore

Hi!

We came across some oddities regarding the certificate handling in SSL_KEY_EXCHANGE protocol. I believe two improvements could be made in this area, as outlined below.

Background

SSL_KEY_EXCHANGE is used in conjunction with ASYM_ENCRYPT in this use case. The keystore on disk contains a leaf certificate (client & server flag set) and the chain up to and including the CA (like this one).

Certificates have a short validity period and are periodically re-generated before expiry. The keystore is re-created when certificates are re-generated. 

The coordinator acts as the key server until it steps down from being a coordinator. The server socket remains open and accepts connections from other peers whenever the view changes. When a peer connects to the server, it verifies that the certificate presented by the server is valid. If it's valid, the peer is able to proceed and fetch the asymmetric key.

Support reloading keystore from disk

The current code uses the same user-provided keystore file as both a keystore and a truststore. When a peer connects to the server, it notices that the server presents the same certificate as the one that resides in the peer's truststore. The peer will in this case "simply" pass the validation, without checking the certificate validity. In other words, if both the server and peer have the same certificate in the keystore, it does not matter that the certificate has expired. However, if the peer updates the keystore (new certificate), it will have a different certificate in it's keystore compared to the key server. A real validation kicks in, and the server certificate is rejected due to being expired.

A simple (perhaps naïve) solution would be to simply re-create the server socket at some point in time. The keystore would be reloaded and new certificate would be taken into use. We could, for example, in handleView() restart the keyserver when the socket has been open for a long time. This would cause the socket factory to create a new socket, reloading the content of the keystore.

Allow user to provide explicit truststore

Related to above, it's a bit unconventional to set the leaf certificate in the truststore, although it could be a valid use case. As we can see, the peer does not use the CA to verify the server-provided certificate, if that same certificate resides in the peer's truststore.

I would like to propose a new set of configuration properties (truststore_name, truststore_type, truststore_password at least) that would allow the user to provide two different keystore files. For backwards compatibility reasons, we can still set the truststore the old way, if truststore_ parameters are not given.

Does it make sense? Happy to help with the implementation if you agree!