Details

    • Steps to Reproduce:
      Hide

      See description

      Show
      See description
    • Workaround Description:
      Hide

      Workaround is the try again at a later time. I placed the retry in a loop, checking every second, however, the GET_PHYSICAL_ADDRESS request worked at the first retry, 1 second later.

      Show
      Workaround is the try again at a later time. I placed the retry in a loop, checking every second, however, the GET_PHYSICAL_ADDRESS request worked at the first retry, 1 second later.

      Description

      Hi, I have a problem with the ViewChanged notification providing a 'wrong' address. In general, addresses are provided in the following format, e.g., ip-10-81-128-189-29581, but occassionally we see UUID like addresses 0fbd6496-9b37-f2ca-3c93-590d6b54324e. We are then attempting to convert this to an IP address using JGroups, which fails

      DefaultCacheManager dcm = (DefaultCacheManager)CacheManager.getInstance().getCacheManager();
      JGroupsTransport jgt = (JGroupsTransport)dcm.getTransport();
      Object addr = jgt.getChannel().getProtocolStack().down(new Event(Event.GET_PHYSICAL_ADDRESS, ((JGroupsAddress)jgroupsAddress).getJGroupsAddress()));

      This is occurring in a cluster with 3+ members, and when a member joins, one other member of the cluster will often get an address as above. Subsequently, if another node joins the cluster, the cluster view that originally received the UUID address will then have the address of that node (represented by the UUID) in the normal format.

      Is this a bug? if not, how can the UUID address be converted to an IP address?

      Our environment is:
      CentOS 6.2
      We are using a GossipRouter to manage our cluster.
      Infinispan 5.1.3.FINAL (JGroups 3.0.8.FINAL).
      Java 1.6.30

      I know that this is occurring via Infinispan, but it looks like a JGroups issue.

      Our gossip router config is:

      <?xml version="1.0" encoding="UTF-8"?>
      <config>
      <TCP bind_port="7900"/>
      <TCPGOSSIP timeout="3000" initial_hosts="10.81.0.227[8800]" num_initial_members="3"/>
      <MERGE2 max_interval="30000" min_interval="10000"/>
      <FD_SOCK/>
      <FD timeout="10000" max_tries="5"/>
      <VERIFY_SUSPECT timeout="1500"/>
      <pbcast.NAKACK use_mcast_xmit="false" retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true"/>
      <UNICAST timeout="300,600,1200,2400,3600"/>
      <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000"/>
      <pbcast.GMS print_local_addr="true" join_timeout="3000" view_bundling="true"/>
      <UFC max_credits="2000000" min_threshold="0.10"/>
      <MFC max_credits="2000000" min_threshold="0.10"/>
      <FRAG2 frag_size="60000"/>
      </config>

      Here is some pared down logging from our servers:

      10.81.0.227

      2012-04-11 22:17:58,123 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-128-189-29581 [NODE|ATTACHED]
      2012-04-11 22:17:58,125 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-128-189-29581 => 10.81.128.189

      2012-04-11 22:18:20,054 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-208-97-19626 [NODE|DETACHED]
      2012-04-11 22:18:20,073 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-19626 => 10.81.208.97

      2012-04-11 22:18:45,515 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-208-97-20403 [NODE|ATTACHED]
      2012-04-11 22:18:45,518 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-20403 => 10.81.208.97

      10.81.128.189

      2012-04-11 22:18:04,686 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-128-189-29581 [NODE|ATTACHED]
      2012-04-11 22:18:04,689 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-128-189-29581 => 10.81.128.189

      2012-04-11 22:18:20,402 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-208-97-19626 [NODE|DETACHED]
      2012-04-11 22:18:20,404 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-19626 => 10.81.208.97

      2012-04-11 22:18:45,640 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - 0fbd6496-9b37-f2ca-3c93-590d6b54324e [NODE|ATTACHED]
      2012-04-11 22:18:45,640 [notification-thread-0] INFO [JmsCluster.getInetAddress() 437] - getInetAddress 0fbd6496-9b37-f2ca-3c93-590d6b54324e
      2012-04-11 22:18:45,641 [notification-thread-0] INFO [JmsCluster.getInetAddress() 441] - getInetAddress, DefaultCacheManager view: members: [ip-10-81-0-227-2930, ip-10-81-128-189-29581, 0fbd6496-9b37-f2ca-3c93-590d6b54324e]
      2012-04-11 22:18:45,641 [notification-thread-0] INFO [JmsCluster.getInetAddress() 444] - getInetAddress, JGroupsTransport view: members: [ip-10-81-0-227-2930, ip-10-81-128-189-29581, 0fbd6496-9b37-f2ca-3c93-590d6b54324e], addresses: [10.81.128.189:7800]
      2012-04-11 22:18:45,642 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress 0fbd6496-9b37-f2ca-3c93-590d6b54324e => null

      2012-04-11 22:37:05,509 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-0-227-2930 [NODE|DETACHED]
      2012-04-11 22:37:05,512 [notification-thread-0] INFO [JmsCluster.getInetAddress() 437] - getInetAddress ip-10-81-0-227-2930
      2012-04-11 22:37:05,512 [notification-thread-0] INFO [JmsCluster.getInetAddress() 441] - getInetAddress, DefaultCacheManager view: members: [ip-10-81-128-189-29581, ip-10-81-208-97-20403]
      2012-04-11 22:37:05,513 [notification-thread-0] INFO [JmsCluster.getInetAddress() 444] - getInetAddress, JGroupsTransport view: members: [ip-10-81-128-189-29581, ip-10-81-208-97-20403], addresses: [10.81.128.189:7800]
      2012-04-11 22:37:05,514 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-0-227-2930 => 10.81.0.227

      the later logging shows that 0fbd6496-9b37-f2ca-3c93-590d6b54324e represents ip-10-81-208-97-20403

      10.81.208.97

      2012-04-11 22:18:53,257 [notification-thread-0] INFO [JmsCluster.viewChanged() 375] - ip-10-81-208-97-20403 [NODE|ATTACHED]
      2012-04-11 22:18:53,260 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-208-97-20403 => 10.81.208.97

      2012-04-11 22:37:18,178 [notification-thread-0] INFO [JmsCluster.viewChanged() 388] - ip-10-81-128-189-29581 [NODE|DETACHED]
      2012-04-11 22:37:18,180 [notification-thread-0] INFO [JmsCluster.getInetAddress() 457] - getInetAddress ip-10-81-128-189-29581 => 10.81.128.189

      Looks very like JGRP-1293, JGRP-1252

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                belaban Bela Ban
                Reporter:
                mackerman Mitchell Ackerman
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: