Using a custom HandlerResolver on client while using WS-Security does not add ws-security header to soap envelope

I have a webservice deployed as an EJB endpoint with the correct WS-Security Configuration (using UsernameToken)

My standalone java client code looks like this:

ReflectorEjbWsseApi port = null;
Service service = (Service) Service.create(wsdlURL, serviceQName);
service.setHandlerResolver(new CustomHandlerResolver);

port = service.getPort(ReflectorEjbWsseApi.class);
HandlerResolver hr = new HandlerResolver();
((BindingProvider)port).getRequestContext().put(BindingProvider.USERNAME_PROPERTY, WebServiceConstants.USERNAME);
((BindingProvider)port).getRequestContext().put(BindingProvider.PASSWORD_PROPERTY, WebServiceConstants.PASSWORD);

URL securityURL = new File("jboss-wsse-client.xml").toURL();
((StubExt)port).setSecurityConfig(securityURL.toExternalForm());

((StubExt) port).setConfigName("Standard WSSecurity Client");
((BindingProvider)port).getRequestContext().put(StubExt.PROPERTY_AUTH_TYPE, StubExt.PROPERTY_AUTH_TYPE_WSSE);

As you can see I am setting my own custom HandlerResolver. However by doing this I am somehow overriding the default HandlerChain which would have used the USERNAME_PROPERTY and PASSWORD_PROPERTY values and added the ws-security header to the soap message.

When executing this code I get a SOAPFault from the server saying that the soap message does not contain a required header.

javax.xml.ws.soap.SOAPFaultException: org.jboss.ws.core.CommonSOAPFaultException: This service requires <wsse:Security>, which is missing.

If I don't set the handlerResolver by removing this line from my client:
service.setHandlerResolver(new CustomHandlerResolver);

the request goes through fine and I get a valid response.

My own customhandlers do nothing but some logging activity. I should be able to use my own custom handlers with WS-Security.