Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-569

CVO History Pruner is non-functional, letting history length above MaxHistory

    XMLWordPrintable

Details

    • Moderate
    • 1
    • OTA 224
    • 1
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:
      Latest implementation of history pruner (pr805 [1]) had increased max upgrade history in cvo to 100, and implemented a weight based pruning priority strategy for in case history length grows any larger. This pruning however is not happening, letting history grow uncontrollably, and potentially reach resource limits of etcd or kubernetes.

      Observed the following while running continuous upgrade-rollback cycles:

      $ oc get clusterversion version -o json | jq '.status.history|length'
      203

      Version-Release number of selected component (if applicable):
      4.12.0-0.nightly-2022-08-23-223922
      4.12.0-0.nightly-2022-08-23-153511

      How reproducible:
      1/1

      Steps to Reproduce:
      Same as described in bz2097067 [2], with addition of waiting a few minutes after the first rollback to allow it to reach 'Completed' state.

      Actual results:
      History grows uncontrollably

      Expected results:
      History should be pruned to keep max size of 100

      Additional info:

      [1] https://github.com/openshift/cluster-version-operator/pull/805
      [2] https://bugzilla.redhat.com/show_bug.cgi?id=2097067#c4

      Attachments

        Issue Links

          Activity

            People

              evakhoni@redhat.com Evgeni Vakhonin
              evakhoni@redhat.com Evgeni Vakhonin
              Evgeni Vakhonin Evgeni Vakhonin
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: