Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-11535

Cluster fails to merge if instances started simultaneously

XMLWordPrintable

    • Hide

      Configure cluster of 4 Keycloak Docker containers, using standalone-ha.xml. Start all four simultaneously. Watch the logs. The Received cluster messages remain at four separate nodes, rather than merging into one cluster of four nodes.
      Also sometimes happens if you have an existing working cluster, and restart the nodes two at a time waiting for each node to finish startup logging before starting the next two.
      The workaround is to stop all instances. Start one instance and wait for it to finish, then start the remaining three.

      Show
      Configure cluster of 4 Keycloak Docker containers, using standalone-ha.xml. Start all four simultaneously. Watch the logs. The Received cluster messages remain at four separate nodes, rather than merging into one cluster of four nodes. Also sometimes happens if you have an existing working cluster, and restart the nodes two at a time waiting for each node to finish startup logging before starting the next two. The workaround is to stop all instances. Start one instance and wait for it to finish, then start the remaining three.

      Our Keycloak docker cluster has four instances, clustered using Jgroups/Infinispan as per the standalone-ha.xml. If you start them all simultaneously the "Receive new cluster" logs indicate four separate clusters, each with a single member. They never get merged into the proper single cluster of four members. It seems to be the merging that has changed. The application then fails (we are not using sticky sessions, and each member is ignorant of the sessions on the other members).

      We can only start the cluster by first starting one instance, then when it is running, starting the other three. The logs then indicate the creation of a single cluster which subsequent instances join.

      This is consistent behaviour, and when we revert back to v.4.5.0, the issue goes away, Unfortunately we need 4.6.0 for an important fix.

      On 4.5.0 we get the message "Received new, MERGED cluster view for channel ejb: MergeView::" when it detects a number of subgroups to merge. This never appears in 4.6.0.

        1. docker-entrypoint.sh
          0.1 kB
        2. Dockerfile
          1 kB
        3. entrypoint.sh
          0.4 kB
        4. standalone-ha.xml
          34 kB
        5. standalone-ha.xml
          37 kB

              rhn-engineering-rhusar Radoslav Husar
              ianrodgers Ian Rodgers (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: