Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-33720

Undiagnosed panic detected in openshift-console-operator

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Major Major
    • 4.15.z
    • 4.16
    • Management Console
    • None
    • Important
    • No
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, the console operator health check controller had a missing return statement, which caused the operator to crash unexpectedly in some cases. With this release, the issue has been fixed. (link:https://issues.redhat.com/browse/OCPBUGS-33720[*OCPBUGS-33720*])
      __________
      The console operator health check controller had a missing return statement, which caused the operator to crash unexpectedly in some cases. The health check controller was updated to return where expected, which fixed the unexpected panic.
      Show
      * Previously, the console operator health check controller had a missing return statement, which caused the operator to crash unexpectedly in some cases. With this release, the issue has been fixed. (link: https://issues.redhat.com/browse/OCPBUGS-33720 [* OCPBUGS-33720 *]) __________ The console operator health check controller had a missing return statement, which caused the operator to crash unexpectedly in some cases. The health check controller was updated to return where expected, which fixed the unexpected panic.
    • Bug Fix
    • Done

      This is a clone of issue OCPBUGS-33505. The following is the description of the original issue:

      Noticed in k8s 1.30 PR, here's the run where it happened:
      https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_kubernetes/1953/pull-ci-openshift-kubernetes-master-e2e-aws-ovn-fips/1788800196772106240

      E0510 05:58:26.315444       1 runtime.go:79] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
      goroutine 992 [running]:
      k8s.io/apimachinery/pkg/util/runtime.logPanic({0x26915e0?, 0x471dff0})
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:75 +0x85
      k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0x0?})
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:49 +0x6b
      panic({0x26915e0?, 0x471dff0?})
      	/usr/lib/golang/src/runtime/panic.go:914 +0x21f
      github.com/openshift/console-operator/pkg/console/controllers/healthcheck.(*HealthCheckController).CheckRouteHealth.func2()
      	/go/src/github.com/openshift/console-operator/pkg/console/controllers/healthcheck/controller.go:156 +0x62
      k8s.io/client-go/util/retry.OnError.func1()
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/client-go/util/retry/util.go:51 +0x30
      k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection(0x2fdcde8?)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:145 +0x3e
      k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff({0x989680, 0x3ff0000000000000, 0x3fb999999999999a, 0x5, 0x0}, 0x2fdcde8?)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:461 +0x5a
      k8s.io/client-go/util/retry.OnError({0x989680, 0x3ff0000000000000, 0x3fb999999999999a, 0x5, 0x0}, 0x2667a00?, 0xc001b185d0?)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/client-go/util/retry/util.go:50 +0xa5
      github.com/openshift/console-operator/pkg/console/controllers/healthcheck.(*HealthCheckController).CheckRouteHealth(0xc001b097e8?, {0x2fdce90?, 0xc00057c870?}, 0x16?, 0x2faf140?)
      	/go/src/github.com/openshift/console-operator/pkg/console/controllers/healthcheck/controller.go:152 +0x9a
      github.com/openshift/console-operator/pkg/console/controllers/healthcheck.(*HealthCheckController).Sync(0xc000748ae0, {0x2fdce90, 0xc00057c870}, {0x7f84e80672b0?, 0x7f852f941108?})
      	/go/src/github.com/openshift/console-operator/pkg/console/controllers/healthcheck/controller.go:143 +0x8eb
      github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile(0xc000b57950, {0x2fdce90, 0xc00057c870}, {0x2fd5350?, 0xc001b185a0?})
      	/go/src/github.com/openshift/console-operator/vendor/github.com/openshift/library-go/pkg/controller/factory/base_controller.go:201 +0x43
      github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem(0xc000b57950, {0x2fdce90, 0xc00057c870})
      	/go/src/github.com/openshift/console-operator/vendor/github.com/openshift/library-go/pkg/controller/factory/base_controller.go:260 +0x1b4
      github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1({0x2fdce90, 0xc00057c870})
      	/go/src/github.com/openshift/console-operator/vendor/github.com/openshift/library-go/pkg/controller/factory/base_controller.go:192 +0x89
      k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:259 +0x22
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x30?)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:226 +0x33
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0014b7b60?, {0x2faf040, 0xc001b18570}, 0x1, 0xc0014b7b60)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:227 +0xaf
      k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00057c870?, 0x3b9aca00, 0x0, 0x0?, 0x0?)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:204 +0x7f
      k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext({0x2fdce90, 0xc00057c870}, 0xc00139c770, 0x0?, 0x0?, 0x0?)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:259 +0x93
      k8s.io/apimachinery/pkg/util/wait.UntilWithContext(...)
      	/go/src/github.com/openshift/console-operator/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:170
      github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker(0x0?, {0x2fdce90?, 0xc00057c870?})
      	/go/src/github.com/openshift/console-operator/vendor/github.com/openshift/library-go/pkg/controller/factory/base_controller.go:183 +0x4d
      github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2()
      	/go/src/github.com/openshift/console-operator/vendor/github.com/openshift/library-go/pkg/controller/factory/base_controller.go:117 +0x65
      created by github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run in goroutine 749
      	/go/src/github.com/openshift/console-operator/vendor/github.com/openshift/library-go/pkg/controller/factory/base_controller.go:112 +0x2ba
      

            rh-ee-jonjacks Jon Jackson
            openshift-crt-jira-prow OpenShift Prow Bot
            YaDan Pei YaDan Pei
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: