Uploaded image for project: 'WildFly WIP'
  1. WildFly WIP
  2. WFWIP-649

KUBE_PING handling of IPv6 on dual-stack clusters

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Blocker
    • Clustering, OpenShift
    • None

    Description

      OpenShift 4.13 dual-stack cluster
      EAP 7.4.12 image from EAP7-1992 (details)

      Cluster network setup:

      Name:         cluster
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  config.openshift.io/v1
Kind:         Network
Metadata:
  Creation Timestamp:  2023-08-02T12:11:49Z
  Generation:          2
  Managed Fields:
    API Version:  config.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        f:clusterNetwork:
        f:externalIP:
          f:policy:
        f:networkType:
        f:serviceNetwork:
      f:status:
        f:clusterNetwork:
        f:clusterNetworkMTU:
        f:networkType:
        f:serviceNetwork:
    Manager:      cluster-network-operator/operconfig
    Operation:    Apply
    Time:         2023-08-02T12:13:17Z
    API Version:  config.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:clusterNetwork:
        f:externalIP:
          .:
          f:policy:
        f:networkType:
        f:serviceNetwork:
      f:status:
    Manager:         cluster-bootstrap
    Operation:       Update
    Time:            2023-08-02T12:11:49Z
  Resource Version:  3099
  UID:               62d21fdc-767c-4cd6-a8e7-a440da32e610
Spec:
  Cluster Network:
    Cidr:         10.128.0.0/14
    Host Prefix:  23
    Cidr:         fd00:10:128::/56
    Host Prefix:  64
  External IP:
    Policy:
  Network Type:  OVNKubernetes
  Service Network:
    172.122.0.0/16
    fd00:172:16::/112
Status:
  Cluster Network:
    Cidr:               10.128.0.0/14
    Host Prefix:        23
    Cidr:               fd00:10:128::/56
    Host Prefix:        64
  Cluster Network MTU:  1400
  Network Type:         OVNKubernetes
  Service Network:
    172.122.0.0/16
    fd00:172:16::/112
Events:  <none>

      Simple application built with SERVER_USE_IPV6=true

      KUBE_PING configuration (I'm aware these options are deprecated):

      OPENSHIFT_KUBE_PING_NAMESPACE=msimka-namespace
OPENSHIFT_KUBE_PING_LABEL=Cluster=Smoke

      Servers don't form cluster
      Pod-1:

      13:13:57,351 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 79) ISPN000094: Received new cluster view for channel ee: [eap-server-1-sh8ks|0] (1) [eap-server-1-sh8ks]
13:13:57,357 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 78) ISPN000079: Channel ee local address is eap-server-1-sh8ks, physical addresses are [fd00:10:128:3:0:0:0:32%eth0:7600]

      Pod-2

      13:13:56,620 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 80) ISPN000094: Received new cluster view for channel ee: [eap-server-1-mjkq6|0] (1) [eap-server-1-mjkq6]
13:13:56,627 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 79) ISPN000079: Channel ee local address is eap-server-1-mjkq6, physical addresses are [fd00:10:128:5:0:0:0:581%eth0:7600]

      from debug log, I can see that KUBE_PING sends discovery requests only to IPv4

      14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) Determining pod status
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.phase=Succeeded
14:40:14,354 DEBUG [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) pod eap-server-1-deploy, group null
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) Determining pod status
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.phase=Succeeded
14:40:14,354 DEBUG [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) pod eap-server-2-deploy, group null
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) Determining pod status
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.phase=Succeeded
14:40:14,354 DEBUG [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) pod eap-server-3-5nrr6, group eap-server-3
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) Determining pod status
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.phase=Running
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.message=null and status.reason=null
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   containerStatuses[].status of all container is true
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) conditions with type=="Ready" has status property value = true
14:40:14,354 DEBUG [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) pod eap-server-3-deploy, group null
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) Determining pod status
14:40:14,354 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.phase=Succeeded
14:40:14,354 DEBUG [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) pod eap-server-3-s45bw, group eap-server-3
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) Determining pod status
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.phase=Running
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   status.message=null and status.reason=null
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6)   containerStatuses[].status of all container is true
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) conditions with type=="Ready" has status property value = true
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) getPods(msimka-namespace, null) = [Pod{name='eap-server-1-build', ip='10.129.3.148', podGroup='null'}, Pod{name='eap-server-1-deploy', ip='10.129.3.149', podGroup='null'}, Pod{name='eap-server-2-deploy', ip='10.129.3.151', podGroup='null'}, Pod{name='eap-server-3-5nrr6', ip='10.131.0.53', podGroup='eap-server-3'}, Pod{name='eap-server-3-deploy', ip='10.129.3.154', podGroup='null'}, Pod{name='eap-server-3-s45bw', ip='10.129.3.155', podGroup='eap-server-3'}]
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) eap-server-3-5nrr6: hosts fetched from Kubernetes: [Pod{name='eap-server-1-build', ip='10.129.3.148', podGroup='null'}, Pod{name='eap-server-1-deploy', ip='10.129.3.149', podGroup='null'}, Pod{name='eap-server-2-deploy', ip='10.129.3.151', podGroup='null'}, Pod{name='eap-server-3-5nrr6', ip='10.131.0.53', podGroup='eap-server-3'}, Pod{name='eap-server-3-deploy', ip='10.129.3.154', podGroup='null'}, Pod{name='eap-server-3-s45bw', ip='10.129.3.155', podGroup='eap-server-3'}]
14:40:14,355 TRACE [org.jgroups.protocols.kubernetes.KUBE_PING] (thread-11,ee,eap-server-3-5nrr6) eap-server-3-5nrr6: sending discovery requests to [10.129.3.148:7600, 10.129.3.149:7600, 10.129.3.151:7600, 10.131.0.53:7600, 10.129.3.154:7600, 10.129.3.155:7600]

      The pod has

              "podIP": "10.131.0.53",
        "podIPs": [
            {
                "ip": "10.131.0.53"
            },
            {
                "ip": "fd00:10:128:3::34"
            }
        ]

      KUBE_PING uses only podIP - https://github.com/jgroups-extras/jgroups-kubernetes/blob/main/src/main/java/org/jgroups/protocols/kubernetes/Client.java#L165

      Attachments

        Activity

          People

            rhn-engineering-rhusar Radoslav Husar
            msimka@redhat.com Martin Simka
            Martin Simka Martin Simka
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated: