Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-21721

[4.13] Redundant rolebinding default-account-openshift-machine-config-operator in manifest

XMLWordPrintable

    • Important
    • No
    • Rejected
    • True
    • Hide

      It results in upgradeable=false after 4.12 to 4.13 upgrade as resource deletions are in progress. So it would block 4.14 upgrade with the clusters born in 4.12 and earlier, as well as 4.14 EUS upgrade

      Show
      It results in upgradeable=false after 4.12 to 4.13 upgrade as resource deletions are in progress. So it would block 4.14 upgrade with the clusters born in 4.12 and earlier, as well as 4.14 EUS upgrade

      Description of problem:

      In the 4.12 to 4.13 upgrade, I'm seeing rbac was re-creating rolebinding default-account-openshift-machine-config-operator after CVO deleted it. Looking at the 4.13 manifests, # grep -r default-account-openshift-machine-config-operator manifests/
      manifests/0000_80_machine-config-operator_03_rbac.yaml:  name: default-account-openshift-machine-config-operator
      manifests/0000_90_machine-config-operator_90_deletion.yaml:  name: default-account-openshift-machine-config-operator
      
      We have 2 manifest files containing rolebinding default-account-openshift-machine-config-operator. 
      
      # cat manifests/0000_90_machine-config-operator_90_deletion.yaml
      # keep default for upgrading purposes, need to delete the existing role:
      # https://github.com/openshift/enhancements/blob/master/dev-guide/cluster-version-operator/dev/object-deletion.md
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: default-account-openshift-machine-config-operator
        annotations:
          include.release.openshift.io/ibm-cloud-managed: "true"
          include.release.openshift.io/self-managed-high-availability: "true"
          include.release.openshift.io/single-node-developer: "true"
          release.openshift.io/delete: "true"
      subjects:
      - kind: ServiceAccount
        name: default
        namespace: openshift-machine-config-operator
      roleRef:
        kind: ClusterRole
        name: cluster-admin
        apiGroup: rbac.authorization.k8s.io
      
      # cat manifests/0000_80_machine-config-operator_03_rbac.yaml
      ---
      kind: ClusterRoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: default-account-openshift-machine-config-operator
        annotations:
          include.release.openshift.io/ibm-cloud-managed: "true"
          include.release.openshift.io/self-managed-high-availability: "true"
          include.release.openshift.io/single-node-developer: "true"
      subjects:
      - kind: ServiceAccount
        name: default
        namespace: openshift-machine-config-operator
      roleRef:
        kind: ClusterRole
        name: cluster-admin
        apiGroup: rbac.authorization.k8s.io
      ---
      ...
      
      

      Version-Release number of selected component (if applicable):

      4.13.0-0.nightly-2023-10-14-180128
      

      How reproducible:

      1/1
      

      Steps to Reproduce:

      1. Install a 4.12 cluster
      2. Upgrade to 4.13
      3.
      

      Actual results:

      rbac was re-creating rolebinding default-account-openshift-machine-config-operator after CVO deleted it
      

      Expected results:

      Either rolebinding default-account-openshift-machine-config-operator is kept or deleted during upgrade
      

      Additional info:

      CVO log:
      yangyang:logs/ $ grep "default-account-openshift-machine-config-operator" current.log | grep -v 'throt\|syn' 
      2023-10-16T07:50:11.793218573Z I1016 07:50:11.793011       1 helper.go:65] Delete requested for clusterrolebinding "default-account-openshift-machine-config-operator".
      2023-10-16T07:50:27.174318546Z I1016 07:50:27.174296       1 upgradeable.go:276] Resource deletions in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"
      2023-10-16T07:50:27.174318546Z I1016 07:50:27.174307       1 upgradeable.go:92] Upgradeability condition failed (type='UpgradeableDeletesInProgress' reason='ResourceDeletesInProgress' message='Cluster minor level upgrades are not allowed while resource deletions are in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"')
      2023-10-16T07:53:49.857739011Z I1016 07:53:49.857719       1 rbac.go:20] ClusterRoleBinding default-account-openshift-machine-config-operator not found, creating
      2023-10-16T07:54:24.198266419Z I1016 07:54:24.198225       1 helper.go:116] Delete of clusterrolebinding "default-account-openshift-machine-config-operator" has already been requested.
      2023-10-16T07:56:12.484953693Z I1016 07:56:12.484945       1 upgradeable.go:276] Resource deletions in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"
      2023-10-16T07:56:12.484969666Z I1016 07:56:12.484961       1 upgradeable.go:92] Upgradeability condition failed (type='UpgradeableDeletesInProgress' reason='ResourceDeletesInProgress' message='Cluster minor level upgrades are not allowed while resource deletions are in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"')
      2023-10-16T07:58:51.702458434Z I1016 07:58:51.702408       1 helper.go:116] Delete of clusterrolebinding "default-account-openshift-machine-config-operator" has already been requested.
      2023-10-16T08:00:08.288021985Z I1016 08:00:08.287915       1 upgradeable.go:276] Resource deletions in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"
      2023-10-16T08:00:08.288021985Z I1016 08:00:08.287923       1 upgradeable.go:92] Upgradeability condition failed (type='UpgradeableDeletesInProgress' reason='ResourceDeletesInProgress' message='Cluster minor level upgrades are not allowed while resource deletions are in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"')
      2023-10-16T08:03:11.306674558Z I1016 08:03:11.306628       1 helper.go:116] Delete of clusterrolebinding "default-account-openshift-machine-config-operator" has already been requested.
      2023-10-16T08:04:04.091715354Z I1016 08:04:04.091697       1 upgradeable.go:276] Resource deletions in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"
      2023-10-16T08:04:04.091720359Z I1016 08:04:04.091715       1 upgradeable.go:92] Upgradeability condition failed (type='UpgradeableDeletesInProgress' reason='ResourceDeletesInProgress' message='Cluster minor level upgrades are not allowed while resource deletions are in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"')
      2023-10-16T08:07:34.716291970Z I1016 08:07:34.715748       1 helper.go:116] Delete of clusterrolebinding "default-account-openshift-machine-config-operator" has already been requested.
      2023-10-16T08:07:59.894596953Z I1016 08:07:59.894580       1 upgradeable.go:276] Resource deletions in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"
      2023-10-16T08:07:59.894610428Z I1016 08:07:59.894597       1 upgradeable.go:92] Upgradeability condition failed (type='UpgradeableDeletesInProgress' reason='ResourceDeletesInProgress' message='Cluster minor level upgrades are not allowed while resource deletions are in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"')
      2023-10-16T08:11:55.698060791Z I1016 08:11:55.698049       1 upgradeable.go:276] Resource deletions in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"
      2023-10-16T08:11:55.698066857Z I1016 08:11:55.698059       1 upgradeable.go:92] Upgradeability condition failed (type='UpgradeableDeletesInProgress' reason='ResourceDeletesInProgress' message='Cluster minor level upgrades are not allowed while resource deletions are in progress; resources=clusterrolebinding "default-account-openshift-machine-config-operator"')
      2023-10-16T08:11:58.368609754Z I1016 08:11:58.368564       1 helper.go:116] Delete of clusterrolebinding "default-account-openshift-machine-config-operator" has already been requested.
      
      

            trking W. Trevor King
            yanyang@redhat.com Yang Yang
            Rio Liu Rio Liu
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: