Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-23456

WMCO in custom namespace in CrashLoopBackOff after upgrade

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • 4.14
    • Windows Containers
    • None
    • Important
    • No
    • 3
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      After upgrading from version 8.x.x to version 9.0.0 in a custom namespace the operator is in CrashLoopBackOff for several minutes  

      Version-Release number of selected component (if applicable):

      upgrading from 8.0.1-d0be5e8 to 9.0.0-9456533

      How reproducible:

      100% on all platforms

      Steps to Reproduce:

      1. install WMCO with 2 Windows machineset nodes on 4.13 version 8.0.1-d0be5e8
      2. install a BYOH node
      3. upgrade the cluster to latest version 4.14
      4. change the catatlogsource wmco index version to latest 9.0.0-9456533 'oc edit catalogsources.operators.coreos.com wmco -n openshift-marketplace'

      Actual results:

      oc logs deployment.apps/windows-machine-config-operator -n winc-namespace-test -f
      {"level":"info","ts":"2023-11-20T10:59:55Z","logger":"version","msg":"operator","version":"8.0.1-d0be5e8"}
      {"level":"info","ts":"2023-11-20T10:59:55Z","logger":"version","msg":"go","version":"go1.19.9 linux/amd64"}
      {"level":"info","ts":"2023-11-20T10:59:55Z","logger":"leader","msg":"Trying to become the leader."}
      {"level":"info","ts":"2023-11-20T10:59:55Z","logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
      {"level":"info","ts":"2023-11-20T10:59:55Z","logger":"leader","msg":"Continuing as the leader."}
      {"level":"info","ts":"2023-11-20T10:59:55Z","logger":"controller-runtime.metrics","msg":"Metrics server is starting to listen","addr":"0.0.0.0:9182"}
      {"level":"error","ts":"2023-11-20T10:59:55Z","logger":"controller.secret","msg":"Unable to retrieve private key, please ensure it is created","error":"the cache is not started, can not read objects","stacktrace":"github.com/openshift/windows-machine-config-operator/controllers.(*SecretReconciler).SetupWithManager\n\t/remote-source/build/windows-machine-config-operator/controllers/secret_controller.go:65\nmain.main\n\t/remote-source/build/windows-machine-config-operator/cmd/operator/main.go:191\nruntime.main\n\t/usr/lib/golang/src/runtime/proc.go:250"}
      {"level":"error","ts":"2023-11-20T10:59:55Z","logger":"setup","msg":"unable to create ConfigMap reconciler","error":"error creating ignition object: failed to parse MachineConfig ignition: unsupported config version\nReport: ","stacktrace":"main.main\n\t/remote-source/build/windows-machine-config-operator/cmd/operator/main.go:201\nruntime.main\n\t/usr/lib/golang/src/runtime/proc.go:250"}
      oc get pods -n winc-namespace-test
      NAME                                               READY   STATUS             RESTARTS        AGE
      windows-machine-config-operator-7fc4988459-mcxrp   0/1     CrashLoopBackOff   8 (4m45s ago)   20m
      
      

      Expected results:

      Operator should continue working an no crashes after upgrade

      Additional info:

      After several minutes the operator resume and the Windows upgrade taking place

              team-winc Team WinC
              rrasouli Aharon Rasouli
              Aharon Rasouli Aharon Rasouli
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: