JGroups Address UUID problem

Hi, I have a problem with the ViewChanged notification providing a 'wrong' address. In general, addresses are provided in the following format, e.g., ip-10-81-128-189-29581, but occassionally we see UUID like addresses 0fbd6496-9b37-f2ca-3c93-590d6b54324e. We are then attempting to convert this to an IP address using JGroups, which fails

DefaultCacheManager dcm = (DefaultCacheManager)CacheManager.getInstance().getCacheManager();
JGroupsTransport jgt = (JGroupsTransport)dcm.getTransport();
Object addr = jgt.getChannel().getProtocolStack().down(new Event(Event.GET_PHYSICAL_ADDRESS, ((JGroupsAddress)jgroupsAddress).getJGroupsAddress()));

This is occurring in a cluster with 3+ members, and when a member joins, one other member of the cluster will often get an address as above. Subsequently, if another node joins the cluster, the cluster view that originally received the UUID address will then have the address of that node (represented by the UUID) in the normal format.

Is this a bug? if not, how can the UUID address be converted to an IP address?

Our environment is:
CentOS 6.2
We are using a GossipRouter to manage our cluster.
Infinispan 5.1.3.FINAL (JGroups 3.0.8.FINAL).
Java 1.6.30

I know that this is occurring via Infinispan, but it looks like a JGroups issue.

Our gossip router config is:

<?xml version="1.0" encoding="UTF-8"?>
<config>
<TCP bind_port="7900"/>
<TCPGOSSIP timeout="3000" initial_hosts="10.81.0.227[8800]" num_initial_members="3"/>
<MERGE2 max_interval="30000" min_interval="10000"/>
<FD_SOCK/>
<FD timeout="10000" max_tries="5"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK use_mcast_xmit="false" retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000"/>
<pbcast.GMS print_local_addr="true" join_timeout="3000" view_bundling="true"/>
<UFC max_credits="2000000" min_threshold="0.10"/>
<MFC max_credits="2000000" min_threshold="0.10"/>
<FRAG2 frag_size="60000"/>
</config>

Here is some pared down logging from our servers:

10.81.0.227

2012-04-11 22:17:58,123 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-128-189-29581 [NODE|ATTACHED]
2012-04-11 22:17:58,125 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-128-189-29581 => 10.81.128.189

2012-04-11 22:18:20,054 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-208-97-19626 [NODE|DETACHED]
2012-04-11 22:18:20,073 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-19626 => 10.81.208.97

2012-04-11 22:18:45,515 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-208-97-20403 [NODE|ATTACHED]
2012-04-11 22:18:45,518 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-20403 => 10.81.208.97

10.81.128.189

2012-04-11 22:18:04,686 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-128-189-29581 [NODE|ATTACHED]
2012-04-11 22:18:04,689 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-128-189-29581 => 10.81.128.189

2012-04-11 22:18:20,402 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-208-97-19626 [NODE|DETACHED]
2012-04-11 22:18:20,404 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-19626 => 10.81.208.97

2012-04-11 22:18:45,640 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - 0fbd6496-9b37-f2ca-3c93-590d6b54324e [NODE|ATTACHED]
2012-04-11 22:18:45,640 [notification-thread-0] INFO [JmsCluster.getInetAddress() 437] - getInetAddress 0fbd6496-9b37-f2ca-3c93-590d6b54324e
2012-04-11 22:18:45,641 [notification-thread-0] INFO [JmsCluster.getInetAddress() 441] - getInetAddress, DefaultCacheManager view: members: [ip-10-81-0-227-2930, ip-10-81-128-189-29581, 0fbd6496-9b37-f2ca-3c93-590d6b54324e]
2012-04-11 22:18:45,641 [notification-thread-0] INFO [JmsCluster.getInetAddress() 444] - getInetAddress, JGroupsTransport view: members: [ip-10-81-0-227-2930, ip-10-81-128-189-29581, 0fbd6496-9b37-f2ca-3c93-590d6b54324e], addresses: [10.81.128.189:7800]
2012-04-11 22:18:45,642 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress 0fbd6496-9b37-f2ca-3c93-590d6b54324e => null

2012-04-11 22:37:05,509 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-0-227-2930 [NODE|DETACHED]
2012-04-11 22:37:05,512 [notification-thread-0] INFO [JmsCluster.getInetAddress() 437] - getInetAddress ip-10-81-0-227-2930
2012-04-11 22:37:05,512 [notification-thread-0] INFO [JmsCluster.getInetAddress() 441] - getInetAddress, DefaultCacheManager view: members: [ip-10-81-128-189-29581, ip-10-81-208-97-20403]
2012-04-11 22:37:05,513 [notification-thread-0] INFO [JmsCluster.getInetAddress() 444] - getInetAddress, JGroupsTransport view: members: [ip-10-81-128-189-29581, ip-10-81-208-97-20403], addresses: [10.81.128.189:7800]
2012-04-11 22:37:05,514 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-0-227-2930 => 10.81.0.227

the later logging shows that 0fbd6496-9b37-f2ca-3c93-590d6b54324e represents ip-10-81-208-97-20403

10.81.208.97

2012-04-11 22:18:53,257 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-208-97-20403 [NODE|ATTACHED]
2012-04-11 22:18:53,260 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-20403 => 10.81.208.97

2012-04-11 22:37:18,178 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-128-189-29581 [NODE|DETACHED]
2012-04-11 22:37:18,180 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-128-189-29581 => 10.81.128.189

Looks very like JGRP-1293, JGRP-1252