Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-65866

unstable ipv6 cidr in k8s.ovn.org/host-cidrs node annotation

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:
      Node.metadata.annotations.k8s.ovn.org/host-cidrs looses its IPv6 element periodically.
      Watching my cluster nodes, I see k8s.ovn.org/host-cidrs=["10.46.248.62/21","2620:52:0:2ef8:e2eb:b796:c5e8:c5d7/64"] for ~1-2 minutes, followed by ~30-60 seconds absence. It seems that different nodes loose their ipv6 cidr concurrently.

      This may have noticeable effect on watchers of the node resource, and may be a sign of a deeper issue in ovnk handles IPv6.

      Version-Release number of selected component (if applicable):
      4.20.1

      How reproducible:
      100% on my cnv.engineering.redhat.com cluster.

      Steps to Reproduce:

      1. poll oc get nodes -o yaml |grep k8s.ovn.org/host-cidrs

      Actual results:
      Within a few minutes, the IPv6 portion of the annotation comes and go. Note that the actual IPv6 address of the hosts, as reported by NodeNetworkStatus, remains stable.

      Expected results:
      Stable annotation value

      Additional info:

      Please fill in the following template while reporting a bug and provide as much relevant information as possible. Doing so will give us the best chance to find a prompt resolution.

      Affected Platforms:
      Bare-metal cluster https://cnv.engineering.redhat.com/
      Please contact #cnv-engineering-users if you need cluster-reader rights to debug this.

      Let me point to Claud-generated scripts that helped me analyze the bug:
      https://gist.github.com/dankenigsberg/697aa2f8dd402338aee15cafd6b408ff
      https://gist.github.com/dankenigsberg/bab35296fb50f0196d68137db845ae40

       

      must-gather of the relevant cluster: https://drive.google.com/file/d/1G7ZfkayLY37xSGQbJKn15X3iyP-v4E-H/view?usp=sharing

              sdn-team-bot sdn-team bot
              dkenigsb@redhat.com Dan Kenigsberg
              None
              None
              Anurag Saxena Anurag Saxena
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: