Uploaded image for project: 'HornetQ'
  1. HornetQ
  2. HORNETQ-721

Message Redistribution based on consumer statistics


    • Type: Feature Request
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.2.0.GA
    • Fix Version/s: 3.0
    • Component/s: None
    • Labels:
    • Affects:
    • Estimated Difficulty:


      Hi guys,

      Thanks for a great product in HornetQ! I have a (fairly common) use case that doesn't appear to be supported, based on conversations I've had in the forum.

      I have a cluster of 3 nodes. Each node is running JBoss 6, using JMS and MDBs. Some nodes are faster than others [5]. In response to a FTP upload, one of the nodes will parse the FTP'd file and produce messages on to the JMS queue. I want those messages consumed as fast as possible by the MDBs on the nodes.

      As of HornetQ 2.2.2.Final, the situation is:

      1. By default HornetQ will server-side load balance the messages. If there are 300 messages, each node will get 100 each. The MDBs on the fast nodes will consume their 100 messages quickly, then sit idle while the slow node consumes its 100 messages.
      2. If I set <max-hops>0</max-hops> I can disable the server-side load balance so all 300 messages go to the one node. But only that node's MDB does any processing. The MDBs on the other nodes sit idle.
      3. I can point all MDBs at the same remote queue. This is most similar to how JBoss Messaging did it, but is not really in the 'spirit' of HornetQ (i.e. goes back to Single Point of Failure)

      Really I want one of these options:

      1. Client side load balancing of MDBs. This is covered in the User Guide [1] for non-MDBs, but apparently is not supported for MDBs ('subsequent sessions created using a single session factory' does not apply)
      2. Redistribution for MDBs. Again, this is covered in the User Guide [2] for non-MDBs, but not supported for MDBs (because they never 'close their last consumer')
      3. Server-side load balancing of the produced messages. Apparently this is not pluggable [3] and even if it were the current API does not seem flexible enough [4] to, say, send two thirds of messages to the fast nodes and only one third to the slow node.



      [1] http://hornetq.sourceforge.net/docs/hornetq-2.0.0.GA/user-manual/en/html/clusters.html
      [2] http://hornetq.sourceforge.net/docs/hornetq-2.0.0.GA/user-manual/en/html/clusters.html
      [3] http://community.jboss.org/thread/167995?tstart=0
      [4] http://community.jboss.org/thread/168040
      [5] This is because we have grown our cluster over several months, as our traffic increased. Each time we bought a new server it tended to be faster than the previous ones. We cannot afford to throw away all our servers and buy new ones every time we want to add a node to our cluster, just for the sake of having an 'even' cluster

        Gliffy Diagrams




              • Assignee:
                clebert.suconic Clebert Suconic
                kennardconsulting Richard Kennard
              • Votes:
                5 Vote for this issue
                4 Start watching this issue


                • Created: