-
Bug
-
Resolution: Done-Errata
-
Undefined
-
None
-
4.14
-
Moderate
-
No
-
CFE Sprint 254
-
1
-
False
-
Description of problem:
cert-manager logs show `fatal error: concurrent map read and map write`
Version-Release number of selected component (if applicable):
OCP 4.14.1 cert-manager-operator-bundle:v1.13.0-3
How reproducible:
Not sure. Hit once
Steps to Reproduce:
1. Install cert-manager from console 2. Check pods: $ oc get po -n cert-manager NAME                    READY  STATUS   RESTARTS     AGE cert-manager-b94b579f-m6j9v         1/1   Running  1 (5m28s ago)  6m20s cert-manager-cainjector-849684c4cd-ldqrw  1/1   Running  0        6m57s cert-manager-webhook-7d946df88f-4lscp    1/1   Running  0        6m59s 3. oc logs --previous cert-manager-b94b579f-m6j9v -n cert-manager > logs/cert-manager-b94b579f-m6j9v_previous.log I1206 13:52:20.156106 1 controller.go:251] "cert-manager/controller/build-context: configured acme dns01 nameservers" nameservers=["172.30.0.10:53"] W1206 13:52:20.156232 1 client_config.go:618] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I1206 13:52:20.157341 1 controller.go:72] "cert-manager/controller: enabled controllers: [certificaterequests-approver certificaterequests-issuer-acme certificaterequests-issuer-ca certificaterequests-issuer-selfsigned certificaterequests-issuer-vault certificaterequests-issuer-venafi certificates-issuing certificates-key-manager certificates-metrics certificates-readiness certificates-request-manager certificates-revision-manager certificates-trigger challenges clusterissuers ingress-shim issuers orders]" I1206 13:52:20.157954 1 controller.go:145] "cert-manager/controller: starting leader election" I1206 13:52:20.157980 1 controller.go:93] "cert-manager/controller: starting metrics server" address="[::]:9402" I1206 13:52:20.158088 1 controller.go:138] "cert-manager/controller: starting healthz server" address="[::]:9403" I1206 13:52:20.158571 1 leaderelection.go:250] attempting to acquire leader lease kube-system/cert-manager-controller... I1206 13:52:20.177614 1 leaderelection.go:260] successfully acquired lease kube-system/cert-manager-controller I1206 13:52:20.178751 1 controller.go:215] "cert-manager/controller: starting controller" controller="challenges" I1206 13:52:20.179185 1 controller.go:215] "cert-manager/controller: starting controller" controller="certificaterequests-approver" I1206 13:52:20.179380 1 controller.go:215] "cert-manager/controller: starting controller" controller="certificaterequests-issuer-ca" I1206 13:52:20.179636 1 controller.go:215] "cert-manager/controller: starting controller" controller="certificates-metrics" I1206 13:52:20.179909 1 controller.go:215] "cert-manager/controller: starting controller" controller="certificates-readiness" I1206 13:52:20.180172 1 controller.go:215] "cert-manager/controller: starting controller" controller="certificates-revision-manager" I1206 13:52:20.180277 1 controller.go:192] "cert-manager/controller: not starting controller as it's disabled" controller="gateway-shim" I1206 13:52:20.180389 1 controller.go:215] "cert-manager/controller: starting controller" controller="issuers" fatal error: concurrent map read and map write goroutine 91 [running]: k8s.io/apimachinery/pkg/runtime.(*Scheme).ObjectKinds(0xc0001c59d0, {0x2c883f0?, 0xc000b92140}) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/runtime/scheme.go:267 +0x1ca k8s.io/apimachinery/pkg/runtime.WithVersionEncoder.Encode({{0x2c8ed40, 0xc000342540}, {0x2c8a470, 0xc00013ce10}, {0x2c8a3d0, 0xc0001c59d0}}, {0x2c883f0, 0xc000b92140}, {0x2c79640, 0xc000ad16e0}) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/runtime/helper.go:223 +0x82 k8s.io/apimachinery/pkg/runtime.Encode({0x2c8ecc8, 0xc000ad16b0}, {0x2c883f0, 0xc000b92140}) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/runtime/codec.go:49 +0x64 k8s.io/client-go/rest.(*Request).Body(0xc0009b7c20, {0x26cef20?, 0xc000b92140?}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/rest/request.go:470 +0x48b k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Update(0xc0003433e0, {0x2ca0dc0, 0xc00045a690}, 0xc000b92140, {{{0x0, 0x0}, {0x0, 0x0}}, {0x0, 0x0, ...}, ...}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/kubernetes/typed/coordination/v1/lease.go:136 +0x15f k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Update(0xc0009b6a20, {0x2ca0dc0, 0xc00045a690}, {{0xc0009e27c0, 0x3c}, 0x3c, {{0x973b938, 0xedd027394, 0x409a240}}, {{0xc1543d050a96cc89, ...}}, ...}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/leaderelection/resourcelock/leaselock.go:75 +0x28b k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew(0xc0009b6b40, {0x2ca0dc0, 0xc00045a690}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:368 +0x42f k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:275 +0x25 k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1({0x7f024f546a68, 0x18}) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:109 +0x1b k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext({0x2ca1d48?, 0xc00042e360?}, 0xc000adfa30?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:154 +0x57 k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc00042e360}, 0x70?, 0x1243f85?, 0x7f024f546a68?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:245 +0x38 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc00042e360}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x2ca0d50?, 0xc0006de190?, 0x40dc4a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:274 +0x10d k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x30?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:226 +0x3e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00042e300?, {0x2c7f1e0, 0xc00045a660}, 0x1, 0xc00042e300) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:227 +0xb6 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0006de190?, 0x37e11d600, 0x0, 0x20?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:204 +0x89 k8s.io/apimachinery/pkg/util/wait.Until(...) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:161 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew(0xc0009b6b40, {0x2ca0d50?, 0xc0006de140?}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:271 +0x125 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run(0xc0009b6b40, {0x2ca0d50, 0xc0004a9450}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:214 +0x119 github.com/cert-manager/cert-manager/controller-binary/app.startLeaderElection({0x2ca0d50, 0xc0004a9450}, 0xc0009ad340, {0x2cc26e8, 0xc000503380}, {0x2c9f6e0?, 0xc000061240}, {0xc000a07e50, 0xc000342f40, 0x0}, ...) /remote-source/app/cmd/controller/app/controller.go:373 +0x328 github.com/cert-manager/cert-manager/controller-binary/app.Run.func6() /remote-source/app/cmd/controller/app/controller.go:151 +0x20c golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 1 [runnable]: type:.eq.k8s.io/apimachinery/pkg/runtime/schema.GroupVersionKind(0xc00087f260?, 0xc00087f308?) <autogenerated>:1 +0xac k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypeWithName(0xc0001c59d0, {{0x272e6d2, 0x19}, {0x270850b, 0x7}, {0x2076db2, 0xd}}, {0x2c8a150?, 0x4099d60}) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/runtime/scheme.go:177 +0x32a k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypes(0xc0001c59d0, {{0x272e6d2?, 0x19?}, {0x270850b?, 0xa?}}, {0x406f980?, 0x6, 0x2c8a1a0?}) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/runtime/scheme.go:148 +0x171 k8s.io/apimachinery/pkg/apis/meta/v1.AddToGroupVersion(0xc0001c59d0?, {{0x272e6d2?, 0x2c8da30?}, {0x270850b?, 0x2c8da58?}}) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/register.go:73 +0x20c sigs.k8s.io/gateway-api/apis/v1beta1.addKnownTypes(0x2705a10?) /remote-source/app/cmd/controller/vendor/sigs.k8s.io/gateway-api/apis/v1beta1/zz_generated.register.go:71 +0x20b k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme(...) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/runtime/scheme_builder.go:29 k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme(...) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/runtime/scheme_builder.go:29 github.com/cert-manager/cert-manager/pkg/controller.(*ContextFactory).Build(0xc0009e1800, {0xc00087f8d8?, 0x0?, 0xc00087f8d0?}) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/context.go:334 +0x122 github.com/cert-manager/cert-manager/pkg/controller.(*Builder).Complete(0xc00087f908) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/builder.go:83 +0x6a github.com/cert-manager/cert-manager/pkg/controller/certificate-shim/ingresses.init.0.func1(0xc0004a92c0?) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/certificate-shim/ingresses/controller.go:150 +0x8c github.com/cert-manager/cert-manager/controller-binary/app.Run(0xc0009ad340, 0xc0009ad468?) /remote-source/app/cmd/controller/app/controller.go:202 +0xe2a github.com/cert-manager/cert-manager/controller-binary/app.NewServerCommand.func1({0x2ca0df8?, 0xc0009a8030?}, 0x0?) /remote-source/app/cmd/controller/app/start.go:59 +0x39 github.com/cert-manager/cert-manager/controller-binary/app.newServerCommand.func1(0xc000004600?, {0x2703631?, 0x5?, 0x5?}) /remote-source/app/cmd/controller/app/start.go:106 +0x1f8 github.com/spf13/cobra.(*Command).execute(0xc000004600, {0xc000130130, 0x5, 0x5}) /remote-source/app/cmd/controller/vendor/github.com/spf13/cobra/command.go:940 +0x862 github.com/spf13/cobra.(*Command).ExecuteC(0xc000004600) /remote-source/app/cmd/controller/vendor/github.com/spf13/cobra/command.go:1068 +0x3bd github.com/spf13/cobra.(*Command).Execute(0xc000280700?) /remote-source/app/cmd/controller/vendor/github.com/spf13/cobra/command.go:992 +0x19 main.main() /remote-source/app/cmd/controller/main.go:38 +0x8f goroutine 24 [select]: go.opencensus.io/stats/view.(*worker).start(0xc0000caa00) /remote-source/app/cmd/controller/vendor/go.opencensus.io/stats/view/worker.go:292 +0xad created by go.opencensus.io/stats/view.init.0 /remote-source/app/cmd/controller/vendor/go.opencensus.io/stats/view/worker.go:34 +0x96 goroutine 53 [syscall]: os/signal.signal_recv() /usr/lib/golang/src/runtime/sigqueue.go:152 +0x2f os/signal.loop() /usr/lib/golang/src/os/signal/signal_unix.go:23 +0x19 created by os/signal.Notify.func1.1 /usr/lib/golang/src/os/signal/signal.go:151 +0x2a goroutine 54 [chan receive]: github.com/cert-manager/cert-manager/internal/cmd/util.SetupExitHandler.func1() /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/internal/cmd/util/signal.go:58 +0x45 created by github.com/cert-manager/cert-manager/internal/cmd/util.SetupExitHandler /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/internal/cmd/util/signal.go:56 +0xd7 goroutine 57 [select]: k8s.io/klog/v2.(*flushDaemon).run.func1() /remote-source/app/cmd/controller/vendor/k8s.io/klog/v2/klog.go:1141 +0x11e created by k8s.io/klog/v2.(*flushDaemon).run /remote-source/app/cmd/controller/vendor/k8s.io/klog/v2/klog.go:1137 +0x17b goroutine 56 [select]: github.com/cert-manager/cert-manager/internal/cmd/util.ContextWithStopCh.func1() /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/internal/cmd/util/context.go:30 +0x87 created by github.com/cert-manager/cert-manager/internal/cmd/util.ContextWithStopCh /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/internal/cmd/util/context.go:28 +0xba goroutine 58 [select]: github.com/cert-manager/cert-manager/internal/cmd/util.ContextWithStopCh.func1() /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/internal/cmd/util/context.go:30 +0x87 created by github.com/cert-manager/cert-manager/internal/cmd/util.ContextWithStopCh /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/internal/cmd/util/context.go:28 +0xba goroutine 59 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc00034dd10) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 60 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 61 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 88 [chan receive]: github.com/cert-manager/cert-manager/controller-binary/app.Run.func1() /remote-source/app/cmd/controller/app/controller.go:82 +0x4d golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 89 [IO wait]: internal/poll.runtime_pollWait(0x7f02281c18d8, 0x72) /usr/lib/golang/src/runtime/netpoll.go:306 +0x89 internal/poll.(*pollDesc).wait(0xc000308980?, 0xc000b04ca0?, 0x0) /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc000308980) /usr/lib/golang/src/internal/poll/fd_unix.go:614 +0x2bd net.(*netFD).accept(0xc000308980) /usr/lib/golang/src/net/fd_unix.go:172 +0x35 net.(*TCPListener).accept(0xc000011d10) /usr/lib/golang/src/net/tcpsock_posix.go:148 +0x25 net.(*TCPListener).Accept(0xc000011d10) /usr/lib/golang/src/net/tcpsock.go:297 +0x3d net/http.(*Server).Serve(0xc000a90000, {0x2c9fa70, 0xc000011d10}) /usr/lib/golang/src/net/http/server.go:3059 +0x385 github.com/cert-manager/cert-manager/controller-binary/app.Run.func2() /remote-source/app/cmd/controller/app/controller.go:94 +0x119 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 90 [semacquire]: sync.runtime_Semacquire(0xc0002f0030?) /usr/lib/golang/src/runtime/sema.go:62 +0x27 sync.(*WaitGroup).Wait(0x24750a0?) /usr/lib/golang/src/sync/waitgroup.go:116 +0x4b golang.org/x/sync/errgroup.(*Group).Wait(0xc0004a4000) /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:53 +0x27 github.com/cert-manager/cert-manager/pkg/healthz.(*Server).Start(0xc000a079f0, {0x2ca0d50?, 0xc0004a9450}, {0x2c9fa70?, 0xc000011fb0}) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/healthz/healthz.go:86 +0x12f github.com/cert-manager/cert-manager/controller-binary/app.Run.func5() /remote-source/app/cmd/controller/app/controller.go:139 +0x138 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 46 [IO wait]: internal/poll.runtime_pollWait(0x7f02281c17e8, 0x72) /usr/lib/golang/src/runtime/netpoll.go:306 +0x89 internal/poll.(*pollDesc).wait(0xc000308a00?, 0xc000b0dcf8?, 0x0) /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc000308a00) /usr/lib/golang/src/internal/poll/fd_unix.go:614 +0x2bd net.(*netFD).accept(0xc000308a00) /usr/lib/golang/src/net/fd_unix.go:172 +0x35 net.(*TCPListener).accept(0xc000011fb0) /usr/lib/golang/src/net/tcpsock_posix.go:148 +0x25 net.(*TCPListener).Accept(0xc000011fb0) /usr/lib/golang/src/net/tcpsock.go:297 +0x3d net/http.(*Server).Serve(0xc000a900f0, {0x2c9fa70, 0xc000011fb0}) /usr/lib/golang/src/net/http/server.go:3059 +0x385 github.com/cert-manager/cert-manager/pkg/healthz.(*Server).Start.func1() /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/healthz/healthz.go:74 +0x28 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 47 [chan receive]: github.com/cert-manager/cert-manager/pkg/healthz.(*Server).Start.func2() /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/healthz/healthz.go:80 +0x4d golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 92 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc0000ae960) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 93 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 94 [runnable]: golang.org/x/net/http2.(*ClientConn).RoundTrip(0xc00017f500, 0xc0000c6b00) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:1296 +0x491 golang.org/x/net/http2.(*Transport).RoundTripOpt(0xc0009ce1b0, 0xc0000c6b00, {0xc0?}) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:564 +0x1c5 golang.org/x/net/http2.(*Transport).RoundTrip(...) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:513 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip({0xc0009e4000?}, 0xc0000c6b00?) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:3136 +0x1b net/http.(*Transport).roundTrip(0xc0009e4000, 0xc0000c6a00) /usr/lib/golang/src/net/http/transport.go:548 +0x3ca net/http.(*Transport).RoundTrip(0x250cba0?, 0xc0004513e0?) /usr/lib/golang/src/net/http/roundtrip.go:17 +0x19 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip(0xc0005431d0, 0xc0000c6900) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/transport/round_trippers.go:317 +0x3e5 k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip(0xc0003e1f80, 0xc0000c6800) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/transport/round_trippers.go:168 +0x350 net/http.send(0xc0000c6800, {0x2c7f960, 0xc0003e1f80}, {0x8?, 0x2687e80?, 0x0?}) /usr/lib/golang/src/net/http/client.go:252 +0x5f7 net/http.(*Client).send(0xc000543200, 0xc0000c6800, {0x7f024f546108?, 0x30?, 0x0?}) /usr/lib/golang/src/net/http/client.go:176 +0x9b net/http.(*Client).do(0xc000543200, 0xc0000c6800) /usr/lib/golang/src/net/http/client.go:716 +0x8fb net/http.(*Client).Do(...) /usr/lib/golang/src/net/http/client.go:582 k8s.io/client-go/rest.(*Request).request(0xc0000f90e0, {0x2ca0d88, 0xc000134008}, 0x1?) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/rest/request.go:1023 +0x3a5 k8s.io/client-go/rest.(*Request).Do(0xc0000f90e0, {0x2ca0d88, 0xc000134008}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/rest/request.go:1063 +0xc9 k8s.io/client-go/kubernetes/typed/core/v1.(*events).CreateWithEventNamespace(0xc000342ea0, 0xc0002ef400) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/kubernetes/typed/core/v1/event_expansion.go:59 +0x10c k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Create(0x23f71a0?, 0xc0003774b0?) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/kubernetes/typed/core/v1/event_expansion.go:160 +0x22 k8s.io/client-go/tools/record.recordEvent({0x2c9f1d0?, 0xc000a07e20?}, 0xc0002ef400, {0x0?, 0x0?, 0x0?}, 0x0?, 0xc0003d6138) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:261 +0x16a k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink(0xc000308d00, {0x2c9f1d0, 0xc000a07e20}, 0xc000b94000, 0x1?) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:223 +0x16b k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1(0x0?) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:200 +0x2f k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:327 +0xa2 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 77 [runnable]: golang.org/x/net/http2.(*clientStream).writeRequest(0xc00017f800, 0xc0000c6b00) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:1464 +0xb27 golang.org/x/net/http2.(*clientStream).doRequest(0x0?, 0x0?) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:1326 +0x1e created by golang.org/x/net/http2.(*ClientConn).RoundTrip /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:1232 +0x34a goroutine 78 [select]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:297 +0x1cd created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:280 +0xca goroutine 75 [IO wait]: internal/poll.runtime_pollWait(0x7f02281c16f8, 0x72) /usr/lib/golang/src/runtime/netpoll.go:306 +0x89 internal/poll.(*pollDesc).wait(0xc000308f00?, 0xc0003a7000?, 0x0) /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc000308f00, {0xc0003a7000, 0xc00, 0xc00}) /usr/lib/golang/src/internal/poll/fd_unix.go:167 +0x299 net.(*netFD).Read(0xc000308f00, {0xc0003a7000?, 0xc0003a704c?, 0x155?}) /usr/lib/golang/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc0002fcc28, {0xc0003a7000?, 0xc000b2d878?, 0xc00042da30?}) /usr/lib/golang/src/net/net.go:183 +0x45 crypto/tls.(*atLeastReader).Read(0xc0002f1800, {0xc0003a7000?, 0xc0002f1800?, 0x0?}) /usr/lib/golang/src/crypto/tls/conn.go:788 +0x3d bytes.(*Buffer).ReadFrom(0xc00042db10, {0x2c79820, 0xc0002f1800}) /usr/lib/golang/src/bytes/buffer.go:202 +0x98 crypto/tls.(*Conn).readFromUntil(0xc00042d880, {0x2c7fb80?, 0xc0002fcc28}, 0xbb9?) /usr/lib/golang/src/crypto/tls/conn.go:810 +0xe5 crypto/tls.(*Conn).readRecordOrCCS(0xc00042d880, 0x0) /usr/lib/golang/src/crypto/tls/conn.go:617 +0x116 crypto/tls.(*Conn).readRecord(...) /usr/lib/golang/src/crypto/tls/conn.go:583 crypto/tls.(*Conn).Read(0xc00042d880, {0xc000b24000, 0x1000, 0x0?}) /usr/lib/golang/src/crypto/tls/conn.go:1316 +0x16f bufio.(*Reader).Read(0xc000b18e40, {0xc00018a660, 0x9, 0xc00017f888?}) /usr/lib/golang/src/bufio/bufio.go:237 +0x1bb io.ReadAtLeast({0x2c795c0, 0xc000b18e40}, {0xc00018a660, 0x9, 0x9}, 0x9) /usr/lib/golang/src/io/io.go:332 +0x9a io.ReadFull(...) /usr/lib/golang/src/io/io.go:351 golang.org/x/net/http2.readFrameHeader({0xc00018a660?, 0x9?, 0xc000000000?}, {0x2c795c0?, 0xc000b18e40?}) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/frame.go:237 +0x6e golang.org/x/net/http2.(*Framer).ReadFrame(0xc00018a620) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/frame.go:498 +0x95 golang.org/x/net/http2.(*clientConnReadLoop).run(0xc000b2df98) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:2275 +0x12e golang.org/x/net/http2.(*ClientConn).readLoop(0xc00017f500) /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:2170 +0x6f created by golang.org/x/net/http2.(*Transport).newClientConn /remote-source/app/cmd/controller/vendor/golang.org/x/net/http2/transport.go:821 +0xc1f goroutine 25 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc000294370) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 26 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 27 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 28 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc00060c780) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:276 +0x305 created by k8s.io/client-go/util/workqueue.newDelayingQueue /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:113 +0x22a goroutine 29 [select]: k8s.io/apimachinery/pkg/util/wait.waitForWithContext({0x2ca1d48, 0xc000114de0}, 0xc0003de348, 0x124324a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:205 +0xeb k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc000114de0}, 0x70?, 0x1243f85?, 0x7f024f546a68?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:260 +0x9a k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc000114de0}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x10?, 0x233fd00?, 0x40ca900?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/cache.WaitForCacheSync(0x2ca5c00?, {0xc00060ca80, 0x7, 0xc}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/cache/shared_informer.go:326 +0x88 github.com/cert-manager/cert-manager/pkg/controller.(*controller).Run(0xc000c02480, 0x5, 0xc000114de0) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/controller.go:104 +0xf9 github.com/cert-manager/cert-manager/controller-binary/app.Run.func7() /remote-source/app/cmd/controller/app/controller.go:217 +0xa5 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 114 [select]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:297 +0x1cd created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:280 +0xca goroutine 30 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc000295630) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 31 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 32 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 129 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc00060d200) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:276 +0x305 created by k8s.io/client-go/util/workqueue.newDelayingQueue /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:113 +0x22a goroutine 130 [select]: k8s.io/apimachinery/pkg/util/wait.waitForWithContext({0x2ca1d48, 0xc000114de0}, 0xc00005ad38, 0x124324a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:205 +0xeb k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc000114de0}, 0x70?, 0x1243f85?, 0x7f024f546108?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:260 +0x9a k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc000114de0}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x10?, 0x233fd00?, 0x40ca900?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/cache.WaitForCacheSync(0x2ca5c00?, {0xc0006d6150, 0x1, 0x1}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/cache/shared_informer.go:326 +0x88 github.com/cert-manager/cert-manager/pkg/controller.(*controller).Run(0xc000c50990, 0x5, 0xc000114de0) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/controller.go:104 +0xf9 github.com/cert-manager/cert-manager/controller-binary/app.Run.func7() /remote-source/app/cmd/controller/app/controller.go:217 +0xa5 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 131 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc000295d10) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 132 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 133 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 134 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc00060d860) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:276 +0x305 created by k8s.io/client-go/util/workqueue.newDelayingQueue /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:113 +0x22a goroutine 135 [select]: k8s.io/apimachinery/pkg/util/wait.waitForWithContext({0x2ca1d48, 0xc000114de0}, 0xc00005ad50, 0x124324a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:205 +0xeb k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc000114de0}, 0x70?, 0x1243f85?, 0x7f024f546108?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:260 +0x9a k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc000114de0}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x10?, 0x233fd00?, 0x40ca900?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/cache.WaitForCacheSync(0x2ca5c00?, {0xc000cc2060, 0x4, 0x6}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/cache/shared_informer.go:326 +0x88 github.com/cert-manager/cert-manager/pkg/controller.(*controller).Run(0xc000c92fc0, 0x5, 0xc000114de0) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/controller.go:104 +0xf9 github.com/cert-manager/cert-manager/controller-binary/app.Run.func7() /remote-source/app/cmd/controller/app/controller.go:217 +0xa5 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 79 [select]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:297 +0x1cd created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:280 +0xca goroutine 136 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc0002c8500) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 137 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 138 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 139 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc000cfc000) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:276 +0x305 created by k8s.io/client-go/util/workqueue.newDelayingQueue /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:113 +0x22a goroutine 140 [select]: k8s.io/apimachinery/pkg/util/wait.waitForWithContext({0x2ca1d48, 0xc000114de0}, 0xc0003de360, 0x124324a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:205 +0xeb k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc000114de0}, 0x70?, 0x1243f85?, 0x7f024f546a68?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:260 +0x9a k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc000114de0}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x10?, 0x233fd00?, 0x40ca900?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/cache.WaitForCacheSync(0x2ca5c00?, {0xc0006d7198, 0x1, 0x1}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/cache/shared_informer.go:326 +0x88 github.com/cert-manager/cert-manager/pkg/controller.(*controller).Run(0xc000ce14d0, 0x5, 0xc000114de0) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/controller.go:104 +0xf9 github.com/cert-manager/cert-manager/controller-binary/app.Run.func7() /remote-source/app/cmd/controller/app/controller.go:217 +0xa5 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 115 [select]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:297 +0x1cd created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:280 +0xca goroutine 141 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc0002c8c80) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 142 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 143 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 144 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc000cfc660) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:276 +0x305 created by k8s.io/client-go/util/workqueue.newDelayingQueue /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:113 +0x22a goroutine 145 [select]: k8s.io/apimachinery/pkg/util/wait.waitForWithContext({0x2ca1d48, 0xc000114de0}, 0xc00005ad68, 0x124324a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:205 +0xeb k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc000114de0}, 0x70?, 0x1243f85?, 0x7f024f546108?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:260 +0x9a k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc000114de0}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x10?, 0x233fd00?, 0x40ca900?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/cache.WaitForCacheSync(0x2ca5c00?, {0xc000964b28, 0x3, 0x3}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/cache/shared_informer.go:326 +0x88 github.com/cert-manager/cert-manager/pkg/controller.(*controller).Run(0xc000d1fb90, 0x5, 0xc000114de0) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/controller.go:104 +0xf9 github.com/cert-manager/cert-manager/controller-binary/app.Run.func7() /remote-source/app/cmd/controller/app/controller.go:217 +0xa5 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 80 [select]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:297 +0x1cd created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:280 +0xca goroutine 146 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc0002c9400) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 147 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 148 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 149 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc000cfcd80) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:276 +0x305 created by k8s.io/client-go/util/workqueue.newDelayingQueue /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:113 +0x22a goroutine 150 [select]: k8s.io/apimachinery/pkg/util/wait.waitForWithContext({0x2ca1d48, 0xc000114de0}, 0xc00005ad80, 0x124324a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:205 +0xeb k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc000114de0}, 0x70?, 0x1243f85?, 0x7f024f546108?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:260 +0x9a k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc000114de0}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x10?, 0x233fd00?, 0x40ca900?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/cache.WaitForCacheSync(0x2ca5c00?, {0xc0004a1990, 0x2, 0x2}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/cache/shared_informer.go:326 +0x88 github.com/cert-manager/cert-manager/pkg/controller.(*controller).Run(0xc000d7c1b0, 0x5, 0xc000114de0) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/controller.go:104 +0xf9 github.com/cert-manager/cert-manager/controller-binary/app.Run.func7() /remote-source/app/cmd/controller/app/controller.go:217 +0xa5 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 161 [select]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:297 +0x1cd created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:280 +0xca goroutine 151 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc0002c9b30) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65 created by k8s.io/apimachinery/pkg/watch.NewLongQueueBroadcaster /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/watch/mux.go:93 +0x116 goroutine 152 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 153 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1() /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:320 +0x73 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/record/event.go:318 +0x13d goroutine 154 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc000cfd380) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:276 +0x305 created by k8s.io/client-go/util/workqueue.newDelayingQueue /remote-source/app/cmd/controller/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:113 +0x22a goroutine 155 [select]: k8s.io/apimachinery/pkg/util/wait.waitForWithContext({0x2ca1d48, 0xc000114de0}, 0xc00005ad98, 0x124324a?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:205 +0xeb k8s.io/apimachinery/pkg/util/wait.poll({0x2ca1d48, 0xc000114de0}, 0x70?, 0x1243f85?, 0x7f024f546108?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:260 +0x9a k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext({0x2ca1d48, 0xc000114de0}, 0x40e027?, 0x20?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x49 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil(0x10?, 0x233fd00?, 0x40ca900?) /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:187 +0x3f k8s.io/client-go/tools/cache.WaitForCacheSync(0x2ca5c00?, {0xc0003da060, 0x2, 0x2}) /remote-source/app/cmd/controller/vendor/k8s.io/client-go/tools/cache/shared_informer.go:326 +0x88 github.com/cert-manager/cert-manager/pkg/controller.(*controller).Run(0xc000db6870, 0x5, 0xc000114de0) /remote-source/app/cmd/controller/vendor/github.com/cert-manager/cert-manager/pkg/controller/controller.go:104 +0xf9 github.com/cert-manager/cert-manager/controller-binary/app.Run.func7() /remote-source/app/cmd/controller/app/controller.go:217 +0xa5 golang.org/x/sync/errgroup.(*Group).Go.func1() /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64 created by golang.org/x/sync/errgroup.(*Group).Go /remote-source/app/cmd/controller/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5 goroutine 162 [select]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1() /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:297 +0x1cd created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /remote-source/app/cmd/controller/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:280 +0xca
Actual results:
cert-manager logs show `fatal error: concurrent map read and map write`
Expected results:
No such fatal error
Additional info:
- is related to
-
OCPBUGS-30068 "fatal error: concurrent map read and map write" on Scheme typeToGVK
- New
-
OCPBUGS-30146 fatal error: concurrent map read and map write between Scheme New() and AddKnownTypeWithName()
- Closed
- links to
-
RHEA-2024:128547 cert-manager Operator for Red Hat OpenShift 1.14.0