Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-7431

openshift-marketplace pods with no 'controller: true' ownerReferences

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done-Errata
    • Normal
    • 4.14.0
    • 4.13
    • OLM
    • Moderate
    • X-Files 232, Yamask 233, Zuko 234
    • 3
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Before this update, registry pods were not drained gracefully by the autoscaler if the controller flag was not set to true in the owner references file. With this update, the controller flag is set to true and draining nodes no longer requires a forceful shutdown.
      (link:https://issues.redhat.com/browse/OCPBUGS-7431[*OCPBUGS-7431*])
      Show
      * Before this update, registry pods were not drained gracefully by the autoscaler if the controller flag was not set to true in the owner references file. With this update, the controller flag is set to true and draining nodes no longer requires a forceful shutdown. (link: https://issues.redhat.com/browse/OCPBUGS-7431 [* OCPBUGS-7431 *])
    • Bug Fix
    • Done

    Description

      Description of problem:

      Porting rhbz#2057740 to Jira. Pods without a controller: true entry in ownerReferences are not gracefully drained by the autoscaler (and potentially other drain-library drainers). Checking a recent 4.13 CI run:

      $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.13-e2e-aws-ovn/1625150492994703360/artifacts/e2e-aws-ovn/gather-extra/artifacts/pods.json | jq -r '.items[].metadata | select([(.ownerReferences // [])[] | select(.controller)] | length == 0) | .namespace + " " + .name + " " + (.ownerReferences | tostring)' | grep -v '^\(openshift-etcd\|openshift-kube-apiserver\|openshift-kube-controller-manager\|openshift-kube-scheduler\) ' 
openshift-marketplace certified-operators-fnm5z [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"certified-operators","uid":"4eb36072-7c56-4663-9b5a-fd23cee85432"}]
openshift-marketplace community-operators-nrfl6 [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"community-operators","uid":"0e164593-5656-4592-9915-1a5367a6a548"}]
openshift-marketplace redhat-marketplace-7j7k9 [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"redhat-marketplace","uid":"14b910c4-0e45-4188-ab57-671070b6a9f1"}]
openshift-marketplace redhat-operators-hxhxw [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"redhat-operators","uid":"ca9028e5-affb-4537-81f1-15e3a5129c6e"}]

      Version-Release number of selected component (if applicable):

      At least 4.11 and 4.13 (above). Likely all OpenShift 4.y which have had these openshift-marketplace pods.

      How reproducible:

      100%

      Steps to Reproduce:

      1. Launch a cluster.
      2. Inspect the openshift-marketplace pods with: oc -n openshift-marketplace get -o json pods | jq -r '.items[].metadata | select(.namespace == "openshift-marketplace" and (([.ownerReferences[] | select(.controller == true)]) | length) == 0) | .name + " " + (.ownerReferences | tostring)'

      Actual results:

      certified-operators-fnm5z [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"certified-operators","uid":"4eb36072-7c56-4663-9b5a-fd23cee85432"}]
community-operators-nrfl6 [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"community-operators","uid":"0e164593-5656-4592-9915-1a5367a6a548"}]
redhat-marketplace-7j7k9 [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"redhat-marketplace","uid":"14b910c4-0e45-4188-ab57-671070b6a9f1"}]
redhat-operators-hxhxw [{"apiVersion":"operators.coreos.com/v1alpha1","blockOwnerDeletion":false,"controller":false,"kind":"CatalogSource","name":"redhat-operators","uid":"ca9028e5-affb-4537-81f1-15e3a5129c6e"}]

      Expected results:

      No output.

      Additional info:

      Figuring out which resource to list as the controller is tricky, but there are workarounds, including pointing at the triggering resource or a ClusterOperator as the controller.

      Attachments

        Issue Links

          blocks

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-28228 openshift-marketplace pods with no 'controller: true' ownerReferences

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          causes

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-27826 MCO failed to drain the node due to the custom catalog source pod with no 'controller: true' ownerReferences

          • Undefined - Priority not specified.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-28228 openshift-marketplace pods with no 'controller: true' ownerReferences

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          is duplicated by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-6062 OLM's catalogsource pods cause a need to manually force drain for a worker to reboot

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          links to

          GitHub openshift/operator-framework-olm#460: OCPBUGS-7431: Registry Pod Controller Flag

          Web Link RHSA-2023:5006 OpenShift Container Platform 4.14.z security update

          Activity

            People

              rh-ee-dfranz Daniel Franz
              trking W. Trevor King
              Jian Zhang Jian Zhang
              Michael Peter Michael Peter
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: