observed panic in kube-scheduler:

2024-05-29T07:53:40.874397450Z E0529 07:53:40.873820       1 runtime.go:79] Observed a panic: "integer divide by zero" (runtime error: integer divide by zero)
2024-05-29T07:53:40.874397450Z goroutine 2363 [running]:
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/runtime.logPanic({0x215e8a0, 0x3c7c150})
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/runtime/runtime.go:75 +0x85
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0x0?})
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/runtime/runtime.go:49 +0x6b
2024-05-29T07:53:40.874397450Z panic({0x215e8a0?, 0x3c7c150?})
2024-05-29T07:53:40.874397450Z     runtime/panic.go:770 +0x132
2024-05-29T07:53:40.874397450Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).findNodesThatFitPod(0xc0005b6900, {0x28f4618, 0xc002a97360}, {0x291d688, 0xc00039f688}, 0xc002ac1a00, 0xc0022fc488)
2024-05-29T07:53:40.874397450Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:505 +0xaf0
2024-05-29T07:53:40.874397450Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).schedulePod(0xc0005b6900, {0x28f4618, 0xc002a97360}, {0x291d688, 0xc00039f688}, 0xc002ac1a00, 0xc0022fc488)
2024-05-29T07:53:40.874397450Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:402 +0x31f
2024-05-29T07:53:40.874397450Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).schedulingCycle(0xc0005b6900, {0x28f4618, 0xc002a97360}, 0xc002ac1a00, {0x291d688, 0xc00039f688}, 0xc002a96370, {0xc18dd5a13410037e, 0x72c11612c5e, 0x3d515e0}, ...)
2024-05-29T07:53:40.874397450Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:149 +0x115
2024-05-29T07:53:40.874397450Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).ScheduleOne(0xc0005b6900, {0x28f4618, 0xc000df7ea0})
2024-05-29T07:53:40.874397450Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:111 +0x698
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:259 +0x1f
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00214bee0?)
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:226 +0x33
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00214bf70, {0x28cfa20, 0xc00169e6c0}, 0x1, 0xc000cfd9e0)
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:227 +0xaf
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc001256f70, 0x0, 0x0, 0x1, 0xc000cfd9e0)
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:204 +0x7f
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext({0x28f4618, 0xc000df7ea0}, 0xc0009be200, 0x0, 0x0, 0x1)
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:259 +0x93
2024-05-29T07:53:40.874397450Z k8s.io/apimachinery/pkg/util/wait.UntilWithContext(...)
2024-05-29T07:53:40.874397450Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:170
2024-05-29T07:53:40.874397450Z created by k8s.io/kubernetes/pkg/scheduler.(*Scheduler).Run in goroutine 2386
2024-05-29T07:53:40.874397450Z     k8s.io/kubernetes/pkg/scheduler/scheduler.go:445 +0x119
2024-05-29T07:53:40.876894723Z panic: runtime error: integer divide by zero [recovered]
2024-05-29T07:53:40.876894723Z     panic: runtime error: integer divide by zero
2024-05-29T07:53:40.876894723Z 
2024-05-29T07:53:40.876894723Z goroutine 2363 [running]:
2024-05-29T07:53:40.876894723Z k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0x0?})
2024-05-29T07:53:40.876894723Z     k8s.io/apimachinery/pkg/util/runtime/runtime.go:56 +0xcd
2024-05-29T07:53:40.876929875Z panic({0x215e8a0?, 0x3c7c150?})
2024-05-29T07:53:40.876929875Z     runtime/panic.go:770 +0x132
2024-05-29T07:53:40.876929875Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).findNodesThatFitPod(0xc0005b6900, {0x28f4618, 0xc002a97360}, {0x291d688, 0xc00039f688}, 0xc002ac1a00, 0xc0022fc488)
2024-05-29T07:53:40.876943106Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:505 +0xaf0
2024-05-29T07:53:40.876953277Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).schedulePod(0xc0005b6900, {0x28f4618, 0xc002a97360}, {0x291d688, 0xc00039f688}, 0xc002ac1a00, 0xc0022fc488)
2024-05-29T07:53:40.876962958Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:402 +0x31f
2024-05-29T07:53:40.876973018Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).schedulingCycle(0xc0005b6900, {0x28f4618, 0xc002a97360}, 0xc002ac1a00, {0x291d688, 0xc00039f688}, 0xc002a96370, {0xc18dd5a13410037e, 0x72c11612c5e, 0x3d515e0}, ...)
2024-05-29T07:53:40.877000640Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:149 +0x115
2024-05-29T07:53:40.877000640Z k8s.io/kubernetes/pkg/scheduler.(*Scheduler).ScheduleOne(0xc0005b6900, {0x28f4618, 0xc000df7ea0})
2024-05-29T07:53:40.877011311Z     k8s.io/kubernetes/pkg/scheduler/schedule_one.go:111 +0x698
2024-05-29T07:53:40.877028792Z k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
2024-05-29T07:53:40.877028792Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:259 +0x1f
2024-05-29T07:53:40.877028792Z k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00214bee0?)
2024-05-29T07:53:40.877028792Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:226 +0x33
2024-05-29T07:53:40.877049294Z k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00214bf70, {0x28cfa20, 0xc00169e6c0}, 0x1, 0xc000cfd9e0)
2024-05-29T07:53:40.877058805Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:227 +0xaf
2024-05-29T07:53:40.877068225Z k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc001256f70, 0x0, 0x0, 0x1, 0xc000cfd9e0)
2024-05-29T07:53:40.877088457Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:204 +0x7f
2024-05-29T07:53:40.877088457Z k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext({0x28f4618, 0xc000df7ea0}, 0xc0009be200, 0x0, 0x0, 0x1)
2024-05-29T07:53:40.877099448Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:259 +0x93
2024-05-29T07:53:40.877099448Z k8s.io/apimachinery/pkg/util/wait.UntilWithContext(...)
2024-05-29T07:53:40.877109888Z     k8s.io/apimachinery/pkg/util/wait/backoff.go:170
2024-05-29T07:53:40.877109888Z created by k8s.io/kubernetes/pkg/scheduler.(*Scheduler).Run in goroutine 2386
2024-05-29T07:53:40.877119479Z     k8s.io/kubernetes/pkg/scheduler/scheduler.go:445 +0x119

    4.17

    there are a lot of instances; see https://search.dptools.openshift.org/?search=runtime+error%3A+integer+divide+by+zero&maxAge=24h&context=1&type=all&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

$ podman run -it corbinu/alpine-w3m -dump -cols 200 "https://search.dptools.openshift.org/?search=runtime+error%3A+integer+divide+by+zero&maxAge=24h&context=1&type=all&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job" | grep 'failures match' | sort
openshift-origin-28839-ci-4.16-e2e-azure-ovn-techpreview-serial (all) - 1 runs, 100% failed, 100% of failures match = 100% impact
periodic-ci-openshift-multiarch-master-nightly-4.15-ocp-e2e-aws-ovn-arm64-techpreview-serial (all) - 3 runs, 33% failed, 100% of failures match = 33% impact
periodic-ci-openshift-release-master-ci-4.14-e2e-aws-sdn-techpreview-serial (all) - 3 runs, 33% failed, 300% of failures match = 100% impact
periodic-ci-openshift-release-master-ci-4.14-e2e-azure-sdn-techpreview-serial (all) - 3 runs, 33% failed, 200% of failures match = 67% impact
periodic-ci-openshift-release-master-ci-4.14-e2e-gcp-sdn-techpreview-serial (all) - 3 runs, 100% failed, 67% of failures match = 67% impact
periodic-ci-openshift-release-master-ci-4.17-e2e-aws-ovn-techpreview-serial (all) - 2 runs, 100% failed, 50% of failures match = 50% impact
periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-techpreview-serial (all) - 2 runs, 100% failed, 50% of failures match = 50% impact
periodic-ci-openshift-release-master-ci-4.17-e2e-gcp-ovn-techpreview-serial (all) - 3 runs, 100% failed, 67% of failures match = 67% impact
periodic-ci-openshift-release-master-nightly-4.16-e2e-vsphere-ovn-techpreview-serial (all) - 1 runs, 100% failed, 100% of failures match = 100% impact
periodic-ci-openshift-release-master-nightly-4.17-e2e-vsphere-ovn-techpreview-serial (all) - 2 runs, 100% failed, 50% of failures match = 50% impact
periodic-ci-openshift-release-master-nightly-4.17-fips-payload-scan (all) - 3 runs, 100% failed, 33% of failures match = 33% impact
pull-ci-openshift-api-master-e2e-aws-serial-techpreview (all) - 8 runs, 100% failed, 50% of failures match = 50% impact
pull-ci-openshift-hypershift-main-e2e-kubevirt-azure-ovn (all) - 27 runs, 70% failed, 5% of failures match = 4% impact
pull-ci-openshift-installer-master-e2e-openstack-dualstack-upi (all) - 6 runs, 83% failed, 20% of failures match = 17% impact

    1.
    2.
    3.
    

    see https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-ci-4.17-e2e-aws-ovn-techpreview-serial/1795684524709908480

you need to pull the must-gather and you will find the panic in the openshift-kube-scheduler pod