Option Usage in Remoting Components
===================================
- this document tries to identify:
  - how options are passed down the call stack for connection establishment and channel establishment and
  - which components process them
- we start with top-level components (Endpoint, ConnectionProvider, ...)  and method calls and work down (RemoteConnection, ClientConnectionOpenListener, ServerConnectionOpenListener)
- there are two key handshakes:
  - connction establishment handshake: ClientConnectionOpenListener to ServerConnectionOpenListener
  - channel establishment handshake: RemoteReadListenenr.CHANNEL_OPEN_REQUEST, RemoteReadListenenr.CHANNEL_OPEN_ACK
  
- i've included two appendicies which show the two key ways in which Endpoints may be configured:
  - using Remoting management subsystem to configure the EndpointService and ConnectorService (server side)
  - using wildfly-config.xml to configure the Contextual<Endpoint> (client side)

Part 1: Connection establishment
--------------------------------
- the OptionMap established by EndpointImpl.connect(...,OptionMap,...) is effecively passed down through the call stack to lower-level components  

0. EndpointImpl
- will setup OptionMaps for client-side connection attempts

client											server
------											------
EndpointImpl.<init>									EndpointImpl.<init>
  // options configurable in the default connection option map				// options configurable in the default connection option map
  Option.READ_TIMEOUT	     	 	 	    	   				Option.READ_TIMEOUT
  Option.WRITE_TIMEOUT									Option.WRITE_TIMEOUT
  Option.TCP_KEEPALIVE									Option.TCP_KEEPALIVE
  Option.HEARTBEAT_INTERVAL								Option.HEARTBEAT_INTERVAL
  // options configurable in the per-URI option map					// options configurable in the per-URI option map
  Option.READ_TIMEOUT	     	 	 						Option.READ_TIMEOUT
  Option.WRITE_TIMEOUT									Option.WRITE_TIMEOUT
  Option.TCP_KEEPALIVE									Option.TCP_KEEPALIVE
  Option.HEARTBEAT_INTERVAL								Option.HEARTBEAT_INTERVAL
  Option.IP_TRAFFIC_CLASS								Option.IP_TRAFFIC_CLASS

EndpointImpl.doGetConnection(URI dest, SSLContext sslContext, AuthenticationConfiguration authConfig, boolean connect)
  // list the options used by ConnectionInfo when setting up the OptionMap
  - for each URI, we take an existing URI-specific OptionMap or the defaultConnectionOptionMap if no URI-specific OptionMap entry exists

ConnectionInfo(...,OptionMap,...)
  // list options used here
  - this optionMap is passed through from ConnectionInfo, and will result in a call to EndpointImpl.connect(...,OptionMap,...)

EndpointImpl.connect(...,OptionMap,...)
// this call descends into RemoteConnectionProvider.connect(...,OptionMap,...)

1. RemoteConnectionProvider
- will create a StreamConnection, optionally wrapped in JsseSslConnection(); which occurs depends on Option.SECURE and Option.SSL_ENABLED 
- this is then wrapped in a RemoteConnection(...,OptionMap connectOptions,...))
- then the Connection handshake is called to exchange parameters and authenticate via SASL
- then the RemoteConnectionHandlerFactory is returned to initialize ConnectionImpl
  - RemoteConnectionHandler wires together RemoteConnectionHandler / RemoteReadListrener / RemoteConnection

client											server
-------											------
<init>											<init>
  Option.SECURE										  Option.SECURE
  
connect(...,OptionMap,...)								NetworkServerProvider.createServer(...,OptionMap,...)
  Option.SSL_ENABLED									  Option.SSL_ENABLE
  											  Option.SSL_STARTTLS
											  Option.SSL_CLIENT_AUTH_MODE

2. XnioWorker / NioXnioWorker
- create the StreamConnection and optionally wrap in a JsseSslStreamConnection

client											server
------											------
XnioWorker.Builder									XnioWorker.Builder
Options.WORKER_NAME									Options.WORKER_NAME
Options.WORKER_TASK_CORE_THREADS							Options.WORKER_TASK_CORE_THREADS
Options.WORKER_TASK_MAX_THREADS								Options.WORKER_TASK_MAX_THREADS
Options.WORKER_TASK_KEEPALIVE								Options.WORKER_TASK_KEEPALIVE
Options.THREAD_DAEMON									Options.THREAD_DAEMON
Options.WORKER_IO_THREADS								Options.WORKER_IO_THREADS
Options.WORKER_READ_THREADS								Options.WORKER_READ_THREADS
Options.WORKER_WRITE_THREADS								Options.WORKER_WRITE_THREADS
Options.STACK_SIZE									Options.STACK_SIZE

configurable options:									configurable options:
Options.WORKER_IO_THREADS								Options.WORKER_IO_THREADS
Options.STACK_SIZE									Options.STACK_SIZE

											// tcp and udp variants
// tcp and udp variants									createStreamConnectionServer(SocketAddress bind, ChannelListener acceptListener, OptionMap)
openStreamConnection(dest, openListener, connectOptions)				  createTcpConnectionServer(SocketAddress bind, ChannelListener acceptListener, OptionMap)
- this delegates calls into XnioThread			   		 		  Options.RECEIVE_BUFFER
											  Options.REUSE_ADDRESSES
											  Options.BACKLOG
											  createUdpServer(SocketAddress bind, ChannelListener bindListener, OptionMap)
											  Options.BROADCAST
											  Options.IP_TRAFFIC_CLASS
											  Options.SEND_BUFFER
											  Options.RECEIVE_BUFFER

3. XnioThread / WorkerThread

client
------
- create the TCP level XNIO part
openStreamConnection(SocketAddress bind, SocketAddress destination, ChannelListener openListenenr, ChannelListener bindListener, OptionMap)
  openTcpStreamConnection(SocketAddress bind, SocketAddress destination, ChannelListener openListenenr, ChannelListener bindListener, OptionMap)
  Options.TCP_OOB_INLINE
  Options.TCP_NODELAY
  Options.IP_TRAFFIC_CLASS
  Options.CLOSE_ABORT
  Options.KEEP_ALIVE
  Options.RECEIVE_BUFFER
  Options.REUSE_ADDRESSES
  Options.SEND_BUFFER
  Options.READ_TIMEOUT
  Options.WRITE_TIMEOUT


4. RemoteConnection
- wrap the StreamConnection in a RemoteConnection()

client											server
------											------
RemoteConnection.<init>(...,OptionMap,...)						RemoteConnection.<init>(...,OptionMap,...)
  RemotingOptions.HEARTBEAT_INTERVAL							  RemotingOptions.HEARTBEAT_INTERVAL


5. Connection handshake (exchanging parameters, authentication via SASL)
- perform the Connection establishment handshake between client and server (between ClientConnectionOpenListner and SevrerConnectionOpenListner)

client											server
------											-----
ClientConnectionOpenListener(...,OptionMap,...)						ServerConnectionOpenListener(...,OptionMap,...)
- client side handshake and SASL authentication for a connection			- server side handskae and SASL authentication for a connection
- installs RemoteConnectionHandler and ReadListener   					- installs RemoteReadListener and RemoteConnectionHandler

ClientConnectionOpenListener.<init>	   			       			ServerConnectionOpenListener.<init>
Options.SASL_MECHANISMS									  RemotingOptions.SERVER_NAME
Options.SASL_DISALLOWED_MECHANISMS							  RemotingOptions.MAX_INBOUND_CHANNELS
RemotingOptions.MAX_INBOUND_CHANNELS							  RemotingOptions.DEFAULT_MAX_INBOUND_CHANNELS
RemotingOptions.DEFAULT_MAX_INBOUND_CHANNELS						  RemotingOptions.MAX_OUTBOUND_CHANNELS					
RemotingOptions.MAX_OUTBOUND_CHANNELS							  RemotingOptions.DEFAULT_MAX_OUTBOUND_CHANNELS
RemotingOptions.DEFAULT_MAX_OUTBOUND_CHANNELS						  RemotingOptions.SASL_PROTOCOL
											  RemotingOptions.DEFAULT_SASL_PROTOCOL
											  Options.SASL_PROPERTIES

// NOTE: no more option processing past this point

6. Return a ConnectionHandlerFactory (post-authentication) 
- this factory creates a RemoteConnectionHandler, which ConnectionIMpl delegates to
- it wires together RemoteReadListener/RemoteConnectionhandler and RemoteConnection


Part 2: Channel establishment
--------------------------------
- the option map passed to the openChannel() call is passed through

client											server
------											------
1. call from EJB client NotifyingHandler
ClientServiceHandle.getClientService(Connection c, OptionMap optionMap)

2. call from Connection, which has an OptionMap
// (set by the RemoteConnectionProvider.connect() option map)
Connection.openChannel(String serviceType, OptionMap optionMap)

3. Use connection handle to send "remote" messages to open a channel
RemoteConnectionHandler.open(serviceType, result, optionMap)

3.1 options used when deciding when to open a channel					3.1 options used when deciding when to open an incoming channel
RemotingOptions.MAX_OUTBOUND_CHANNELS   	       	 					RemotingOptions.MAX_INBOUND_CHANNELS

RemoteReadListener send CHANNEL_OPEN_REQUEST			->			RemoteReadListener handleEvent(): CHANNEL_OPEN_REQUEST		

RemoteReadListener handleEvent(): CHANNEL_OPEN_ACK		<-			RemoteReadListener: send CHANNEL_OPEN_ACK

3.2 options passed to the CHANNEL_OPEN_REQUEST
// defaults are taken from RemoteConnection OptionMap
// the final values from OptionMap passed to open() 
RemotingOptions.TRANSMIT_WINDOW_SIZE
RemotingOptions.RECEIVE_WINDOW_SIZE
RemotingOptions.MAX_OUTBOUND_MESSAGES
RemotingOptions.MAX_INBOUND_MESSAGES
RemotingOptions.MAX_OUTBOUND_MESSAGE_SIZE
RemotingOptions.MAX_INBOUND_MESSAGE_SIZE


Appendix I: Remoting subsystem configuration
============================================
- this is used to configure two Remoting abstractions:
  - the server endpoint
  - the connector used to establish the AcceptingChannel<StreamConnection> stream server

<subsystem>
  <endpoint worker=".." authorize-id=".." auth-realm=".." sasl-protocol=".."
            max-outbound-message-size=".." buffer-region-size=".." receive-buffer-size=".." authentication-retries=".." transmit-window-size=".." max-outbound-messages=".."
	    send-buffer-szie=".." max-inbound-messages=".." receive-window-size=".." heartbeat-interval=".." max-inbound-message-size=".."
	    max-outbound-channels=".." max-inbound-channels=".." server-name="..">

  // deprecatred
  <worker-thread-pool read-threads=".." write-threads=".." task-core-threads=".." task-max-threads=".." task-keepalive=".." task-limit=".." >

  // connector configuration - SASL authentication
  <connector name=".." security-realm=".." server-name=".." sasl-protocol=".." sasl-autnentication-factory=".." socket-binding=".." ssl-context="..">
  
    <!-- configuration of the SASL authentication layer for this server -->
    <sasl>
      <include-mechanisms value="..">
      <qop value="auth|auth-int|auth-conf">
      <strength>low|med|high</strength>
      <reuse-session value="true|false">
      <server-auth value="true|false">
      <policy>
        <forward-secrecy value="true|false">
        <no-active value="true|false">
        <no-anonymous value="true|false">
        <no-dictionary value="true|false">
        <no-plain-text value="true|false">
        <pass-credentials value="true|false">
      </policy>
      <!-- general SASL properties -->
      <properties>
        <property name=".." value="..">
      </properties>
    </sasl>
    
    <authentication-provider name="..">

    <!-- properties that end up the the connectorProperties map (only XNIO type) -->
    <properties>
      <property name=".." value="..">
    </properties>
  </connector>
  
  <http-connector connector-ref="..">
    // similar to connector
  </http-connector>

  <outbound-connections>
    <local-outbound-connection>
    <remote-outbound-connection>
    <outbound-connection>
  </outbound-connections>
  
</subsystem>


Appendix II: wildly-config.xml configuration
============================================
- this is used to configure the Contextual<Endpoint> installed on the server

- this file holds common configuration for the following sections:
  <authentication-client>
  <jboss-ejb-client>
  <endpoint>
  <worker>
- we only discuss <endpoint/> and <worker/>

1. Endpoint configuration
- configures a default Contextual<Endpoint>
- general endpoint options used to create defaultConnectionOptionMap
- configured connections create OptionMaps for specific connections
- providers are connection provider implementations

<endpoint name="..." read-timeout="..." write-timeout="..." tcp-keepalive="..." heartbeat-interval="..">

  <providers>
    <!-- connection provider -->
    <provider scheme=".." aliases=".." module=".." class=".."/>
  </providers>
  
  <connections>
    <!-- configured connection options -->
    <connection destination=.." read-timeout="..." write-timeout="..." tcp-keepalive="..." heartbeat-interval=".." ip-traffic-class=".."/>
  </connections>
</endpoint>

2. Worker configuration
- configures a default XNIO worker Contextual<XnioWorker>

<worker>
  <daemon-threads value=".."/>
  <worker-name value=".."/>
  <pool-size max-threads=".."/>
  <task-keepalive value=".."/>
  <io-threads value=".."/>
  <stack-size value=".."/>
  <outbound-bind-addresses>
    <bind-address match=".." bind-address=".." bind-port=".."/>
  </outbound-bind-addresses>
</worker>