Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-18352

winc upgrades are failing from 4.13 -> 4.14 due to remote ovnkube-controller is not ready

    XMLWordPrintable

Details

    • Important
    • No
    • Approved
    • False
    • Hide

      None

      Show
      None
    • Hide
      Cause: ovn-interconnect assumed all nodes had the annotation "k8s.ovn.org/node-subnets".

      Consequence: Since hybrid-overlay nodes do not have the annotation "k8s.ovn.org/node-subnets" if a hybrid-overlay node was present in the cluster the upgrade would fail.

      Fix: skip hybrid-overlay nodes when upgrading

      Result: Upgrade completes successfully
      Show
      Cause: ovn-interconnect assumed all nodes had the annotation "k8s.ovn.org/node-subnets". Consequence: Since hybrid-overlay nodes do not have the annotation "k8s.ovn.org/node-subnets" if a hybrid-overlay node was present in the cluster the upgrade would fail. Fix: skip hybrid-overlay nodes when upgrading Result: Upgrade completes successfully
    • Bug Fix
    • In Progress

    Description

      Description of problem:

      From CLBO ovnkube-node logs:

Upgrade hack: Timed out waiting for the remote ovnkube-controller to be ready even after 5 minutes, err : context deadline exceeded, unable to fetch node-subnet annotation for node ip-10-0-133-201.us-east-2.compute.internal: err, could not find "k8s.ovn.org/node-subnets" annotation

ovnkube-controller not ready implies the absence of node-subnets annotation

CNO upgrade stuck at DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" rollout is not making progress - last change 2023-08-30T10:06:44Z

      Version-Release number of selected component (if applicable):

      4.14.0-0.nightly-2023-08-11-055332

      How reproducible:

      Always

      Steps to Reproduce:

      1.Install OCP cluster with RHCOS and win nodes on 4.13
2.Perform upgrade to 4.14
3.

      Actual results:

      Upgrades failed on CNO

      Expected results:

      Upgrade should pass

      Additional info:

      must-gather: http://shell.lab.bos.redhat.com/~anusaxen/must-gather.local.1473221474492991466/

      Attachments

        Issue Links

          is blocked by

          Spike - Represents a research-related task. SDN-4106 ImpactStatementRequested: OCPBUGS-18352: winc upgrades are failing from 4.13 -> 4.14 due to remote ovnkube-controller is not ready

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-19812 [4.14] winc upgrades are failing from 4.13 -> 4.14 due to remote ovnkube-controller is not ready

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is depended on by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-19812 [4.14] winc upgrades are failing from 4.13 -> 4.14 due to remote ovnkube-controller is not ready

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          links to

          GitHub openshift/ovn-kubernetes#1907: OCPBUGS-17641,OCPBUGS-18352,OCPBUGS-18549,OCPBUGS-19456,OCPBUGS-17455: [DownstreamMerge] 9-26-23

          Web Link RHEA-2023:7198 rpm

          Web Link upstream link

          Activity

            People

              jtanenba@redhat.com Jacob Tanenbaum
              anusaxen Anurag Saxena
              Anurag Saxena Anurag Saxena
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: