It's very hard to test the exception from the client, as there is not even a requirement that the server send a particular error page. However a servlet, installed as the outer most filter, should be able to find the exception is thrown.

I don't know how you check if a new transient conversation is created.