Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-30236

numaresources-controller-manager in CrashLoopBackOff - invalid memory address or nil pointer dereference

XMLWordPrintable

    • Critical
    • No
    • CNF Compute Sprint 251
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Release Note Not Required
    • In Progress

      Description of problem:

      Pod numaresources-controller-manager is in CrashLoopBackOff state
      oc get po -n openshift-numaresources
      NAME                                                READY   STATUS             RESTARTS          AGE
      numaresources-controller-manager-766c55596b-9nb6b   0/1     CrashLoopBackOff   163 (3m52s ago)   14h
      secondary-scheduler-85959757db-dvpdj                1/1     Running            0                 14h
      oc logs -n openshift-numaresources numaresources-controller-manager-766c55596b-9nb6b
      ...
      I0305 07:32:51.102133       1 shared_informer.go:341] caches populated
      I0305 07:32:51.102210       1 controller.go:220] "Starting workers" controller="kubeletconfig" controllerGroup="machineconfiguration.openshift.io" controllerKind="KubeletConfig" worker count=1
      I0305 07:32:51.102295       1 kubeletconfig_controller.go:69] "Starting KubeletConfig reconcile loop" object="/autosizing-master"
      I0305 07:32:51.102412       1 panic.go:884] "Finish KubeletConfig reconcile loop" object="/autosizing-master"
      I0305 07:32:51.102448       1 controller.go:115] "Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference" controller="kubeletconfig" controllerGroup="machineconfiguration.openshift.io" controllerKind="KubeletConfig" KubeletConfig="autosizing-master" namespace="" name="autosizing-master" reconcileID="91d2c547-993c-4ae1-beab-1afc0a72af68"
      panic: runtime error: invalid memory address or nil pointer dereference [recovered]
              panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1778a1c]
      
      goroutine 481 [running]:
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
              /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:116 +0x1fa
      panic({0x19286e0, 0x2d16fc0})
              /usr/lib/golang/src/runtime/panic.go:884 +0x213
      github.com/openshift-kni/numaresources-operator/pkg/kubeletconfig.MCOKubeletConfToKubeletConf(...)
              /remote-source/app/pkg/kubeletconfig/kubeletconfig.go:29
      github.com/openshift-kni/numaresources-operator/controllers.(*KubeletConfigReconciler).reconcileConfigMap(0xc0004d29c0, {0x1e475f0, 0xc000e31260}, 0xc000226c40, {{0x0?, 0xc000e31260?}, {0xc000b98498?, 0x2de08f8?}})
              /remote-source/app/controllers/kubeletconfig_controller.go:126 +0x11c
      github.com/openshift-kni/numaresources-operator/controllers.(*KubeletConfigReconciler).Reconcile(0xc0004d29c0, {0x1e475f0, 0xc000e31260}, {{{0x0, 0x0}, {0xc000b98498, 0x11}}})
              /remote-source/app/controllers/kubeletconfig_controller.go:90 +0x3cd
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1e4a2e0?, {0x1e475f0?, 0xc000e31260?}, {{{0x0?, 0xb?}, {0xc000b98498?, 0x0?}}})
              /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119 +0xc8
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0004446e0, {0x1e47548, 0xc0003520f0}, {0x19b9940?, 0xc00093a1a0?})
              /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316 +0x3ca
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0004446e0, {0x1e47548, 0xc0003520f0})
              /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266 +0x1d9
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
              /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227 +0x85
      created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
              /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:223 +0x587
      

      Version-Release number of selected component (if applicable):

      numaresources-operator.v4.15.0
          

      How reproducible:

      so far 100%
          

      Steps to Reproduce:

          1. Create a KubeletConfig that configures autosizing:
      
      apiVersion: machineconfiguration.openshift.io/v1
      kind: KubeletConfig
      metadata:
        name: autosizing-master
      spec:
        autoSizingReserved: true
        machineConfigPoolSelector:
          matchLabels:
            pools.operator.machineconfiguration.openshift.io/master: ""
      
          2. Create a performance profile that targets subset of nodes
          3. Proceed with numaresources-operator installation
          

      Actual results:

      Pod in CrashLoopBackOff state
          

      Expected results:

      numaresources-operator is successfully installed
          

      Additional info:

      Baremetal dualstack cluster deployed with GitOps-ZTP
          

            fromani@redhat.com Francesco Romani
            yprokule@redhat.com Yurii Prokulevych
            Shereen Haj Shereen Haj
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: