--- contention: cycles/second=2900021037 3215076170383170 36658 @ 0x40c3b8 0x1081a37 0xc05afe 0xc059b6 0xc058a9 0x10819ab 0x1081968 0xc0853a 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1081a36 k8s.io/client-go/tools/cache.(*processorListener).run.func1+0x56 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:968 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x10819aa k8s.io/apimachinery/pkg/util/wait.Until+0x6a k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x1081967 k8s.io/client-go/tools/cache.(*processorListener).run+0x27 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:967 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 3215005817036362 49366 @ 0x451705 0x1081759 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1081758 k8s.io/client-go/tools/cache.(*processorListener).pop+0x118 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:938 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 1706155586308368 1176692 @ 0x40c3b8 0x1025467 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1025466 k8s.io/client-go/util/workqueue.(*Type).updateUnfinishedWorkLoop+0xa6 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:314 1705631387940774 59009 @ 0x451705 0x1022a85 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1022a84 k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop+0x304 k8s.io/client-go@v0.27.4/util/workqueue/delaying_queue.go:276 1604425443330120 12664 @ 0x451705 0x107aec5 0x107862e 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x107aec4 k8s.io/client-go/tools/cache.watchHandler+0x184 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:712 # 0x107862d k8s.io/client-go/tools/cache.(*Reflector).watch+0x54d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:431 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 1587034379496424 15241 @ 0x451705 0x917127 0x9165be 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x917126 golang.org/x/net/http2.(*clientStream).writeRequest+0xb26 golang.org/x/net@v0.12.0/http2/transport.go:1461 # 0x9165bd golang.org/x/net/http2.(*clientStream).doRequest+0x1d golang.org/x/net@v0.12.0/http2/transport.go:1323 1351982155323784 737 @ 0x451705 0x1077ed4 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1077ed3 k8s.io/client-go/tools/cache.(*Reflector).startResync+0x113 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:368 1156662907617434 13681 @ 0x47e26c 0x10245e5 0x1b277cf 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x10245e4 k8s.io/client-go/util/workqueue.(*Type).Get+0xa4 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:200 # 0x1b277ce github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x6e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:246 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1019618573109446 8110 @ 0x47e26c 0x90048b 0x91d895 0x4e4f5a 0x1041388 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x1041387 k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Read+0x87 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:76 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 967623243095894 5055 @ 0x451705 0xc05a35 0xc058a9 0xc05bd9 0x1b268cc 0x1b268a0 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc05a34 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0x134 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:238 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b268cb k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x8b k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b2689f github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func3+0x5f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:141 964820981329886 48467 @ 0x451705 0x92cd75 0x92d485 0x936d0a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x92cd74 google.golang.org/grpc/internal/transport.(*controlBuffer).get+0x114 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:408 # 0x92d484 google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x84 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:535 # 0x936d09 google.golang.org/grpc/internal/transport.newHTTP2Client.func6+0x69 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:442 801281126408622 971 @ 0x47e26c 0x1072cb6 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x1a83f2b 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x1072cb5 k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x255 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:575 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 # 0x1a83f2a k8s.io/client-go/informers.(*sharedInformerFactory).Start.func1+0x6a k8s.io/client-go@v0.27.4/informers/factory.go:150 584799460511622 4551 @ 0x47e26c 0x90048b 0x91d895 0x5f3dc8 0x5f39bb 0x5f35f8 0x1041745 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x5f3dc7 encoding/json.(*Decoder).refill+0x187 encoding/json/stream.go:165 # 0x5f39ba encoding/json.(*Decoder).readValue+0xba encoding/json/stream.go:140 # 0x5f35f7 encoding/json.(*Decoder).Decode+0x77 encoding/json/stream.go:63 # 0x1041744 k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read+0x1a4 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:152 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 418932755445336 697 @ 0x47e26c 0x1072cb6 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x1072cb5 k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x255 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:575 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 238941895096784 5829 @ 0x451705 0xc07bed 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc07bec k8s.io/apimachinery/pkg/util/wait.poller.func1.1+0x1cc k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:297 147114824736600 17561 @ 0x451705 0xc05a35 0xc058a9 0xc05765 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc05a34 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0x134 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:238 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 98655695319446 5103 @ 0x451705 0xc08a4b 0xc0783a 0xc07749 0xc076c6 0x23a5388 0x23a6086 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07748 k8s.io/apimachinery/pkg/util/wait.PollImmediateInfiniteWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:229 # 0xc076c5 k8s.io/apimachinery/pkg/util/wait.PollImmediateInfinite+0x45 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:214 # 0x23a5387 github.com/openshift/library-go/pkg/controller/fileobserver.(*pollingObserver).processReactors+0x87 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/fileobserver/observer_polling.go:86 # 0x23a6085 github.com/openshift/library-go/pkg/controller/fileobserver.(*pollingObserver).Run+0xa5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/fileobserver/observer_polling.go:161 98483981269746 566 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 98483960414806 566 @ 0x47e26c 0x10245e5 0x22cd83e 0x22d7dd2 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x10245e4 k8s.io/client-go/util/workqueue.(*Type).Get+0xa4 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:200 # 0x22cd83d k8s.io/apiserver/pkg/server/dynamiccertificates.(*ConfigMapCAController).processNextWorkItem+0x5d k8s.io/apiserver@v0.27.4/pkg/server/dynamiccertificates/configmap_cafile_content.go:231 # 0x22d7dd1 k8s.io/apiserver/pkg/server/dynamiccertificates.(*ConfigMapCAController).runWorker+0x31 k8s.io/apiserver@v0.27.4/pkg/server/dynamiccertificates/configmap_cafile_content.go:226 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 98483233901774 30945 @ 0x451705 0x90434c 0x902725 0x901bbf 0x8bedb3 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90434b golang.org/x/net/http2.(*serverConn).serve+0x8ab golang.org/x/net@v0.12.0/http2/server.go:938 # 0x902724 golang.org/x/net/http2.(*Server).ServeConn+0xb04 golang.org/x/net@v0.12.0/http2/server.go:531 # 0x901bbe golang.org/x/net/http2.ConfigureServer.func1+0xfe golang.org/x/net@v0.12.0/http2/server.go:321 # 0x8bedb2 net/http.(*conn).serve+0x1292 net/http/server.go:1903 49314310153494 3401 @ 0x451705 0x602e9e 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x602e9d k8s.io/klog/v2.(*flushDaemon).run.func1+0x11d k8s.io/klog/v2@v2.100.1/klog.go:1141 49242430330444 285 @ 0x47e26c 0x10245e5 0x22d5358 0x22d7cb2 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x10245e4 k8s.io/client-go/util/workqueue.(*Type).Get+0xa4 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:200 # 0x22d5357 k8s.io/apiserver/pkg/server/dynamiccertificates.(*DynamicServingCertificateController).processNextWorkItem+0x57 k8s.io/apiserver@v0.27.4/pkg/server/dynamiccertificates/tlsconfig.go:263 # 0x22d7cb1 k8s.io/apiserver/pkg/server/dynamiccertificates.(*DynamicServingCertificateController).runWorker+0x31 k8s.io/apiserver@v0.27.4/pkg/server/dynamiccertificates/tlsconfig.go:258 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 49068514005266 282 @ 0x451705 0xc05a35 0xc058a9 0xc08608 0xc085f5 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc05a34 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0x134 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:238 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc08607 k8s.io/apimachinery/pkg/util/wait.Until+0x27 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0xc085f4 k8s.io/apimachinery/pkg/util/wait.Forever+0x14 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:80 48558654588110 644 @ 0x451705 0xc05a35 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc05a34 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0x134 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:238 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 48546248079174 671 @ 0x451705 0x1f96df2 0x1b2656c 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1f96df1 github.com/openshift/library-go/pkg/operator/status.(*StatusSyncer).watchVersionGetterPostRunHook+0x131 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:256 # 0x1b2656b github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func5+0xab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:156 48541725686742 842 @ 0x47e26c 0x10245e5 0x23de84e 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x10245e4 k8s.io/client-go/util/workqueue.(*Type).Get+0xa4 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:200 # 0x23de84d github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x6d github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:198 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 46980216000492 73 @ 0x47e26c 0x10245e5 0x1f89e2e 0x1f8c58f 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0xc057cb 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x10245e4 k8s.io/client-go/util/workqueue.(*Type).Get+0xa4 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:200 # 0x1f89e2d github.com/openshift/library-go/pkg/operator/configobserver/featuregates.(*defaultFeatureGateAccess).processNextWorkItem+0x6d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/featuregates/simple_featuregate_reader.go:278 # 0x1f8c58e github.com/openshift/library-go/pkg/operator/configobserver/featuregates.(*defaultFeatureGateAccess).runWorker+0x4e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/featuregates/simple_featuregate_reader.go:273 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc057ca k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x2a k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 41760002089500 4 @ 0x451705 0xc08a4b 0xc0783a 0xc07486 0xc073ff 0x2316cf2 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07485 k8s.io/apimachinery/pkg/util/wait.PollUntilWithContext+0x45 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:111 # 0xc073fe k8s.io/apimachinery/pkg/util/wait.PollUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:98 # 0x2316cf1 k8s.io/apiserver/pkg/util/flowcontrol/request.(*objectCountTracker).RunUntil+0x71 k8s.io/apiserver@v0.27.4/pkg/util/flowcontrol/request/object_count_tracker.go:145 24264760024764 220 @ 0x40c3b8 0x1ab2bd3 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1ab2bd2 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0x72 k8s.io/client-go@v0.27.4/tools/record/event.go:320 12135118363616 164 @ 0x40c3b8 0xd29c85 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0xd29c84 k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop+0x64 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:268 5359939842580 22406 @ 0x451705 0x982e73 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x982e72 google.golang.org/grpc.(*ccBalancerWrapper).watcher+0x72 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:115 1085431839148 379 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1082529366216 55 @ 0x47e26c 0x1072cb6 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x226212b 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x1072cb5 k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x255 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:575 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 # 0x226212a k8s.io/client-go/dynamic/dynamicinformer.(*dynamicSharedInformerFactory).Start.func1+0x6a k8s.io/client-go@v0.27.4/dynamic/dynamicinformer/informer.go:108 1001816238742 1008 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x1f00885 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 873750052636 709 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 676753368332 7 @ 0x451705 0xc05a35 0xc058a9 0x20b86fa 0x20b817c 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc05a34 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0x134 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:238 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b86f9 k8s.io/client-go/tools/leaderelection.(*LeaderElector).acquire+0x219 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:246 # 0x20b817b k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x5b k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:203 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 642819318316 193 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x18aa23a 0x1f00fa5 0x1f070db 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa239 k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:81 # 0x1f00fa4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceAccountImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:229 # 0x1f070da github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x71a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:126 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 625449824318 177 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 587759283306 249 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 537701950288 210 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 382173842360 108 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 345420747400 89 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x23c23ff 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23c23fe github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0xde github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:657 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 316072762262 97 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x18a837a 0x1efffde 0x1f06ed9 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a8379 k8s.io/client-go/kubernetes/typed/core/v1.(*services).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:79 # 0x1efffdd github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceImproved+0x15d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:139 # 0x1f06ed8 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x518 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:114 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 293050253564 81 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x1f00885 0x23c49f1 0x23c494c 0x23c03f6 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23c49f0 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1050 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23c494b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).ensureInstallerPod+0xfab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:923 # 0x23c03f5 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xc95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:476 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 279428512366 68 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x1f00885 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 246907106204 78 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1897405 0x1eff858 0x1f06ddf 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897404 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:76 # 0x1eff857 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyNamespaceImproved+0xb7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:92 # 0x1f06dde github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x41e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:108 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 144388443986 4207 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ffa03f 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 138354763444 4472 @ 0x40c378 0x937069 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x937068 google.golang.org/grpc/internal/transport.newHTTP2Client.func4+0x88 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:240 138322098634 2720 @ 0x451705 0x1ff9a10 0x1ff656c 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff656b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x16b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:342 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 136733877726 4473 @ 0x451705 0x7f605b 0xb8277a 0x93502f 0x98c8eb 0x98c7b4 0x98c485 0x98bc25 0x98a005 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x7f605a google.golang.org/grpc/credentials.(*tlsCreds).ClientHandshake+0x2ba google.golang.org/grpc@v1.51.0/credentials/tls.go:90 # 0xb82779 go.etcd.io/etcd/client/v3/credentials.(*transportCredential).ClientHandshake+0x39 go.etcd.io/etcd/client/v3@v3.5.7/credentials/credentials.go:79 # 0x93502e google.golang.org/grpc/internal/transport.newHTTP2Client+0x98e google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:278 # 0x98c8ea google.golang.org/grpc/internal/transport.NewClientTransport+0x38a google.golang.org/grpc@v1.51.0/internal/transport/transport.go:587 # 0x98c7b3 google.golang.org/grpc.(*addrConn).createTransport+0x253 google.golang.org/grpc@v1.51.0/clientconn.go:1269 # 0x98c484 google.golang.org/grpc.(*addrConn).tryAllAddrs+0x4c4 google.golang.org/grpc@v1.51.0/clientconn.go:1213 # 0x98bc24 google.golang.org/grpc.(*addrConn).resetTransport+0x1c4 google.golang.org/grpc@v1.51.0/clientconn.go:1148 # 0x98a004 google.golang.org/grpc.(*addrConn).connect+0x104 google.golang.org/grpc@v1.51.0/clientconn.go:803 59501629584 12 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189001c 0x1f01b30 0x1f03d25 0x1f03d26 0x1f0397a 0x23b61b6 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189001b k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:123 # 0x1f01b2f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x52f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:269 # 0x1f03d24 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x1f03d25 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialConfigMap+0x325 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:492 # 0x1f03979 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncConfigMap+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:455 # 0x23b61b5 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x515 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:236 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 57525791602 16 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x23bde58 0x23beb5c 0x23bf85f 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23beb5b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.nodeToStartRevisionWith+0x2db github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:319 # 0x23bf85e github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xfe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:434 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 44578408422 9 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189efdc 0x1f00db0 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189efdb k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:127 # 0x1f00daf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0x5ef github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:195 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 44075261772 1114 @ 0x451705 0x1ff9a10 0x1ff739d 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff739c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x19c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:412 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 32024406890 4208 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 31520268398 4205 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb206a9 0xb96523 0xb8ee88 0xb8e7f0 0x1ffa1ed 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb206a8 go.etcd.io/etcd/api/v3/etcdserverpb.(*kVClient).Range+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:6460 # 0xb96522 go.etcd.io/etcd/client/v3.(*retryKVClient).Range+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:105 # 0xb8ee87 go.etcd.io/etcd/client/v3.(*kv).Do+0x1e7 go.etcd.io/etcd/client/v3@v3.5.7/kv.go:149 # 0xb8e7ef go.etcd.io/etcd/client/v3.(*kv).Get+0xcf go.etcd.io/etcd/client/v3@v3.5.7/kv.go:119 # 0x1ffa1ec github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x28c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:124 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 26272807470 235 @ 0x451705 0x1ff9a10 0x1ff739d 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff739c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x19c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:412 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 24079282396 83 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x107d648 0x1b255da 0x1b2587d 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x1b255d9 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x119 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:69 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 21976172206 33 @ 0x451705 0x1079165 0x1077acc 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1079164 k8s.io/client-go/tools/cache.(*Reflector).list+0x464 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:524 # 0x1077acb k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x22b k8s.io/client-go@v0.27.4/tools/cache/reflector.go:346 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 21958341442 8 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189e99a 0x23bde58 0x23c25f4 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23c25f3 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0x2d3 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:687 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 20934913380 1588 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 20840777410 4 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189efdc 0x1f00db0 0x23c49f1 0x23c494c 0x23c03f6 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189efdb k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:127 # 0x1f00daf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0x5ef github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:195 # 0x23c49f0 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1050 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23c494b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).ensureInstallerPod+0xfab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:923 # 0x23c03f5 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xc95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:476 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 20739188270 1311 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 19251870280 1588 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x1f00885 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 18588341356 1028 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 18557916442 1300 @ 0x451705 0x874c8b 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x874c8a net/http.setRequestCancel.func4+0x8a net/http/client.go:398 16587198684 1311 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 16469233546 1417 @ 0x451705 0x2320e1e 0x23059a2 0x8bf60f 0x2322ae9 0x8bf60f 0x230730d 0x8bf60f 0x2301be6 0x8bf60f 0x22f5ec9 0x8bf60f 0x229d931 0x2307f6a 0x8bf60f 0x2306ded 0x8bf60f 0x2306aa7 0x8bf60f 0x23053d1 0x8bf60f 0x2324571 0x8bf60f 0x22ff3aa 0x8bf60f 0x238758b 0x8c2756 0x8c57a5 0x90bc23 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x2320e1d k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP+0x23d k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:117 # 0x23059a1 k8s.io/apiserver/pkg/endpoints/filters.withRequestDeadline.func1+0x361 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/request_deadline.go:100 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x2322ae8 k8s.io/apiserver/pkg/server/filters.withWaitGroup.func1+0x1e8 k8s.io/apiserver@v0.27.4/pkg/server/filters/waitgroup.go:86 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x230730c k8s.io/apiserver/pkg/endpoints/filters.WithWarningRecorder.func1+0x18c k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/warning.go:35 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x2301be5 k8s.io/apiserver/pkg/endpoints/filters.WithCacheControl.func1+0x125 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/cachecontrol.go:31 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x22f5ec8 k8s.io/apiserver/pkg/server/httplog.withLogging.func1+0xa8 k8s.io/apiserver@v0.27.4/pkg/server/httplog/httplog.go:111 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x229d930 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*Handler).ServeHTTP+0x1110 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/handler.go:204 # 0x2307f69 k8s.io/apiserver/pkg/endpoints/filters.WithLatencyTrackers.func1+0x269 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:57 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x2306dec k8s.io/apiserver/pkg/endpoints/filters.WithRequestInfo.func1+0x1ec k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/requestinfo.go:39 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x2306aa6 k8s.io/apiserver/pkg/endpoints/filters.withRequestReceivedTimestampWithClock.func1+0x166 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/request_received_time.go:38 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x23053d0 k8s.io/apiserver/pkg/endpoints/filters.WithMuxAndDiscoveryComplete.func1+0x190 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/mux_discovery_complete.go:52 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x2324570 k8s.io/apiserver/pkg/server/filters.withPanicRecovery.func1+0xb0 k8s.io/apiserver@v0.27.4/pkg/server/filters/wrap.go:74 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x22ff3a9 k8s.io/apiserver/pkg/endpoints/filters.withAuditInit.func1+0x289 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/audit_init.go:63 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x238758a k8s.io/apiserver/pkg/server.(*APIServerHandler).ServeHTTP+0x2a k8s.io/apiserver@v0.27.4/pkg/server/handler.go:189 # 0x8c2755 net/http.serverHandler.ServeHTTP+0x315 net/http/server.go:2936 # 0x8c57a4 net/http.initALPNRequest.ServeHTTP+0x244 net/http/server.go:3545 # 0x90bc22 golang.org/x/net/http2.(*serverConn).runHandler+0x82 golang.org/x/net@v0.12.0/http2/server.go:2314 16405115844 1342 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 15630656840 3 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189f6c7 0x23c31b8 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189f6c6 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Delete+0x126 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:169 # 0x23c31b7 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0xe97 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:675 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 14018513544 819 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 11837759794 4531 @ 0x451705 0x555486 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x555485 net.(*netFD).connect.func2+0x85 net/fd_unix.go:118 11265095410 819 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 10667236550 403 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 10421331950 2 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1890274 0x1f02630 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 10420788228 2 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1890274 0x1f02630 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 10407773190 2 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1890274 0x1f02630 0x1f03d25 0x1f03d26 0x1f0397a 0x23b61b6 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x1f03d24 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x1f03d25 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialConfigMap+0x325 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:492 # 0x1f03979 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncConfigMap+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:455 # 0x23b61b5 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x515 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:236 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 9972237564 2 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189001c 0x1f01b30 0x23b5f7f 0x23b5f3a 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189001b k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:123 # 0x1f01b2f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x52f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:269 # 0x23b5f7e github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x2de github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23b5f39 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x299 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:223 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 9610751190 645 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x193e2b4 0x20b6ccb 0x20b962f 0x20b8f25 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x20b8c8d 0xc05afe 0xc059b6 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193e2b3 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:137 # 0x20b6cca k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Update+0x28a k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:75 # 0x20b962e k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x42e k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:363 # 0x20b8f24 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1+0x24 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:270 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x20b8c8c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1+0x10c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:269 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 9226229558 2 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x18a733c 0x1f030b3 0x1f048bc 0x1f048bd 0x1f0437a 0x23b6465 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a733b k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:123 # 0x1f030b2 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecretImproved+0x6f2 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:378 # 0x1f048bb github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecret+0x4bb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:87 # 0x1f048bc github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialSecret+0x4bc github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:571 # 0x1f04379 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncSecret+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:516 # 0x23b6464 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:245 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 9147695856 819 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 8563119846 645 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x193da1a 0x20b6511 0x20b937d 0x20b8f25 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x20b8c8d 0xc05afe 0xc059b6 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193da19 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:78 # 0x20b6510 k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Get+0x90 k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:42 # 0x20b937c k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x17c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:324 # 0x20b8f24 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1+0x24 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:270 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x20b8c8c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1+0x10c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:269 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 8518794944 2 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1890274 0x1f02630 0x23b5f7f 0x23b5f3a 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x23b5f7e github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x2de github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23b5f39 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x299 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:223 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 8128146630 280 @ 0x451705 0x8d38e6 0x8cf7fe 0x8b6799 0x102c4e5 0x873e57 0x8736db 0x8759db 0x23fe4e5 0x23fe4af 0x2404be3 0x240526a 0x240260a 0x2406372 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x8d38e5 net/http.(*Transport).getConn+0x5c5 net/http/transport.go:1382 # 0x8cf7fd net/http.(*Transport).roundTrip+0x79d net/http/transport.go:590 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x23fe4e4 net/http.(*Client).Do+0x164 net/http/client.go:582 # 0x23fe4ae github.com/prometheus/client_golang/api.(*httpClient).Do+0x12e github.com/prometheus/client_golang@v1.14.0/api/client.go:125 # 0x2404be2 github.com/prometheus/client_golang/api/prometheus/v1.(*apiClientImpl).Do+0x42 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:1122 # 0x2405269 github.com/prometheus/client_golang/api/prometheus/v1.(*apiClientImpl).DoGetFallback+0x209 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:1184 # 0x2402609 github.com/prometheus/client_golang/api/prometheus/v1.(*httpAPI).Query+0x389 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:854 # 0x2406371 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller.(*FSyncController).sync+0x151 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller/fsync_controller.go:54 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 8121447498 28 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x107d648 0x1b255da 0x1b2587d 0x23b30e8 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x1b255d9 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x119 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:69 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 8119013188 3 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 8008269382 706 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 7766180478 2 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1890274 0x1f02630 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 7441293404 357 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 7432320250 688 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18aa23a 0x1f00fa5 0x1f070db 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa239 k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:81 # 0x1f00fa4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceAccountImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:229 # 0x1f070da github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x71a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:126 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 7173159806 706 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff6542 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff6541 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x141 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:337 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 6253252692 371 @ 0x451705 0x8daa92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x8daa91 net/http.(*persistConn).writeLoop+0xf1 net/http/transport.go:2410 6236434630 656 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be75a5 0x1f0abeb 0x1f127bf 0x1f07deb 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be75a4 k8s.io/client-go/dynamic.(*dynamicResourceClient).Get+0x544 k8s.io/client-go@v0.27.4/dynamic/simple.go:280 # 0x1f0abea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceMonitor+0x20a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/monitoring.go:60 # 0x1f127be github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyKnownUnstructured+0x1be github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/unstructured.go:18 # 0x1f07dea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x142a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:212 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 5915421936 559 @ 0x451705 0x8db934 0x8cf85a 0x8b6799 0x102c4e5 0x873e57 0x8736db 0x8759db 0x23fe4e5 0x23fe4af 0x2404be3 0x240526a 0x240260a 0x2406372 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x8db933 net/http.(*persistConn).roundTrip+0x993 net/http/transport.go:2638 # 0x8cf859 net/http.(*Transport).roundTrip+0x7f9 net/http/transport.go:603 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x23fe4e4 net/http.(*Client).Do+0x164 net/http/client.go:582 # 0x23fe4ae github.com/prometheus/client_golang/api.(*httpClient).Do+0x12e github.com/prometheus/client_golang@v1.14.0/api/client.go:125 # 0x2404be2 github.com/prometheus/client_golang/api/prometheus/v1.(*apiClientImpl).Do+0x42 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:1122 # 0x2405269 github.com/prometheus/client_golang/api/prometheus/v1.(*apiClientImpl).DoGetFallback+0x209 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:1184 # 0x2402609 github.com/prometheus/client_golang/api/prometheus/v1.(*httpAPI).Query+0x389 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:854 # 0x2406371 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller.(*FSyncController).sync+0x151 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller/fsync_controller.go:54 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 5786607432 234 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb89e96 0xb92ead 0xb93848 0x1ff600f 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb89e95 go.etcd.io/etcd/client/v3.(*Client).Dial+0x235 go.etcd.io/etcd/client/v3@v3.5.7/client.go:254 # 0xb92eac go.etcd.io/etcd/client/v3.NewMaintenance.func1+0x4c go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:81 # 0xb93847 go.etcd.io/etcd/client/v3.(*maintenance).Status+0x67 go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:180 # 0x1ff600e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x12e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:298 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 5222587552 3 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x18a6f50 0x1a4a745 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a6f4f k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:95 # 0x1a4a744 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 4896787162 706 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 4847755328 706 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 4717745362 39 @ 0x451705 0x1ff9a10 0x1ff739d 0x23e1fae 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff739c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x19c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:412 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4670751720 48 @ 0x451705 0x1ff9a10 0x1ff739d 0x23e1fae 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff739c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x19c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:412 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4662017266 706 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 4610491060 706 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 4575101248 1 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x18992b9 0x1a476e7 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18992b8 k8s.io/client-go/kubernetes/typed/core/v1.(*nodes).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/node.go:93 # 0x1a476e6 k8s.io/client-go/informers/core/v1.NewFilteredNodeInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/node.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 4373386652 5 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23ccfde 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23ccfdd github.com/openshift/library-go/pkg/operator/staticpod/controller/node.NodeController.sync+0xd5d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/node/node_controller.go:118 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4064448394 360 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x19c5265 0x1f0d268 0x1f07465 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19c5264 k8s.io/client-go/kubernetes/typed/rbac/v1.(*clusterRoleBindings).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/rbac/v1/clusterrolebinding.go:75 # 0x1f0d267 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyClusterRoleBinding+0xa7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/rbac.go:60 # 0x1f07464 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0xaa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:152 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4060713894 14 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x107d648 0x1b255da 0x1b2587d 0x1f14cd3 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x1b255d9 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x119 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:69 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 # 0x1f14cd2 github.com/openshift/library-go/pkg/operator/resourcesynccontroller.(*ResourceSyncController).Run+0x32 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:95 3976454392 281 @ 0x40c378 0x8d4f65 0x8d5ce6 0x8d4570 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x8d4f64 net/http.(*persistConn).addTLS+0x364 net/http/transport.go:1550 # 0x8d5ce5 net/http.(*Transport).dialConn+0x9e5 net/http/transport.go:1624 # 0x8d456f net/http.(*Transport).dialConnFor+0xaf net/http/transport.go:1456 3866419350 381 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3857503894 328 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1897405 0x1eff858 0x1f06ddf 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897404 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:76 # 0x1eff857 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyNamespaceImproved+0xb7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:92 # 0x1f06dde github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x41e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:108 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3843611516 281 @ 0x451705 0x7c0678 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x7c0677 crypto/tls.(*Conn).handshakeContext.func2+0x97 crypto/tls/conn.go:1467 3620141220 39 @ 0x451705 0x1ff9a10 0x1ff739d 0x23e1fae 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff739c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x19c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:412 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3568354996 55 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f51b7c 0x1f960b0 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f51b7b github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterOperators).UpdateStatus+0x19b github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusteroperator.go:131 # 0x1f960af github.com/openshift/library-go/pkg/operator/status.StatusSyncer.Sync+0x104f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:215 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3523478070 381 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 3352497058 91 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1894cec 0x1895b22 0x1ab1e0a 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1894ceb k8s.io/client-go/kubernetes/typed/core/v1.(*events).CreateWithEventNamespace+0x10b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:59 # 0x1895b21 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Create+0x21 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:160 # 0x1ab1e09 k8s.io/client-go/tools/record.recordEvent+0x169 k8s.io/client-go@v0.27.4/tools/record/event.go:261 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 3326841486 5 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c6b8e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c6b8d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x4cd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1084 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3240441984 12428 @ 0x40b61d 0x10809af 0x108095c 0x107fe2c 0x10702ea 0x107fa96 0x107301e 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x1a83f2b 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x10809ae k8s.io/client-go/tools/cache.(*processorListener).add+0x16e k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x108095b k8s.io/client-go/tools/cache.(*sharedProcessor).distribute+0x11b k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:785 # 0x107fe2b k8s.io/client-go/tools/cache.(*sharedIndexInformer).OnUpdate+0x16b k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:670 # 0x10702e9 k8s.io/client-go/tools/cache.processDeltas+0x249 k8s.io/client-go@v0.27.4/tools/cache/controller.go:457 # 0x107fa95 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HandleDeltas+0x175 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:638 # 0x107301d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x5bd k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:603 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 # 0x1a83f2a k8s.io/client-go/informers.(*sharedInformerFactory).Start.func1+0x6a k8s.io/client-go@v0.27.4/informers/factory.go:150 3014552508 280 @ 0x451705 0x561a45 0x55efd9 0x546dfb 0x547ca8 0x8d2c27 0x8d5b2c 0x8d4570 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x561a44 net.(*Resolver).lookupIPAddr+0x504 net/lookup.go:334 # 0x55efd8 net.(*Resolver).internetAddrList+0x518 net/ipsock.go:288 # 0x546dfa net.(*Resolver).resolveAddrList+0x41a net/dial.go:234 # 0x547ca7 net.(*Dialer).DialContext+0x447 net/dial.go:422 # 0x8d2c26 net/http.(*Transport).dial+0xe6 net/http/transport.go:1176 # 0x8d5b2b net/http.(*Transport).dialConn+0x82b net/http/transport.go:1614 # 0x8d456f net/http.(*Transport).dialConnFor+0xaf net/http/transport.go:1456 2964394236 3 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c0e89 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c0e88 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0x1728 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:501 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2953045954 328 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18a837a 0x1efffde 0x1f06ed9 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a8379 k8s.io/client-go/kubernetes/typed/core/v1.(*services).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:79 # 0x1efffdd github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceImproved+0x15d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:139 # 0x1f06ed8 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x518 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:114 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2903378488 1433 @ 0x40c378 0x5511e5 0x55064f 0x5658a9 0x555c3d 0x56231d 0x53fc9b 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x5511e4 net.(*Resolver).goLookupIPCNAMEOrder.func4+0x84 net/dnsclient_unix.go:659 # 0x55064e net.(*Resolver).goLookupIPCNAMEOrder+0xcee net/dnsclient_unix.go:669 # 0x5658a8 net.(*Resolver).lookupIP+0x188 net/lookup_unix.go:80 # 0x555c3c net.glob..func1+0x3c net/hook.go:23 # 0x56231c net.(*Resolver).lookupIPAddr.func1+0x3c net/lookup.go:326 # 0x53fc9a internal/singleflight.(*Group).doCall+0x3a internal/singleflight/singleflight.go:93 2902684878 10 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x107d648 0x1b255da 0x1b2587d 0x23d7f3e 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x1b255d9 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x119 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:69 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 # 0x23d7f3d github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Run+0xbd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:440 2899446812 1 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x18a85d0 0x1a4ae45 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a85cf k8s.io/client-go/kubernetes/typed/core/v1.(*services).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:96 # 0x1a4ae44 k8s.io/client-go/informers/core/v1.NewFilteredServiceInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/service.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 2814004952 27 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23ccfde 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23ccfdd github.com/openshift/library-go/pkg/operator/staticpod/controller/node.NodeController.sync+0xd5d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/node/node_controller.go:118 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 2641230936 6 @ 0x451705 0x1ff9a10 0x1ff6d17 0x1ff710a 0x23f01f0 0x23ed0a5 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff6d16 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).HealthyMembers+0x156 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:382 # 0x1ff7109 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).HealthyVotingMembers+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:391 # 0x23f01ef github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).getHealthyVotingMembers+0x2f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:447 # 0x23ed0a4 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0x5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:177 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2519368256 345 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 2354705942 345 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2230502558 280 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 2118567632 280 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1786789786 25 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1895249 0x1895be6 0x1ab1d34 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1895248 k8s.io/client-go/kubernetes/typed/core/v1.(*events).PatchWithEventNamespace+0x188 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:99 # 0x1895be5 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Patch+0x25 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:168 # 0x1ab1d33 k8s.io/client-go/tools/record.recordEvent+0x93 k8s.io/client-go@v0.27.4/tools/record/event.go:255 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 1765838946 234 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5f46 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5f45 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x65 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:290 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 1733653804 1 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x189efdc 0x1f00db0 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189efdb k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:127 # 0x1f00daf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0x5ef github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:195 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1521546560 309 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x188fe66 0x1a44845 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x188fe65 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:112 # 0x1a44844 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 1205650022 69 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x1f00885 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1181422968 88 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x1f00885 0x23c49f1 0x23c494c 0x23c03f6 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23c49f0 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1050 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23c494b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).ensureInstallerPod+0xfab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:923 # 0x23c03f5 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xc95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:476 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1115425416 4207 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ffa03f 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 1024423904 6 @ 0x451705 0x1ff9a10 0x1ff656c 0x23ed5a2 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ff9a0f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth+0x32f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:61 # 0x1ff656b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x16b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:342 # 0x23ed5a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0xae1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:199 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 957041178 67 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 954477718 1103 @ 0x481bcb 0x1ae082f 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0x1ae082e github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2+0x2e github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:473 938947450 191 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x18a7186 0x1a4a5e5 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x18a7185 k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:112 # 0x1a4a5e4 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 917120474 91 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x23c23ff 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23c23fe github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0xde github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:657 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 906755204 98 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e94a2 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e94a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x121 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:152 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 896325256 17 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x23bde58 0x23beb5c 0x23bf85f 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23beb5b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.nodeToStartRevisionWith+0x2db github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:319 # 0x23bf85e github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xfe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:434 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 866128772 234 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22f09 0xb97703 0xb938b2 0x1ff600f 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22f08 go.etcd.io/etcd/api/v3/etcdserverpb.(*maintenanceClient).Status+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7323 # 0xb97702 go.etcd.io/etcd/client/v3.(*retryMaintenanceClient).Status+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:202 # 0xb938b1 go.etcd.io/etcd/client/v3.(*maintenance).Status+0xd1 go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:185 # 0x1ff600e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x12e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:298 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 853452946 87 @ 0x47e26c 0x90048b 0x91d895 0x4e4f5a 0x104142d 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x104142c k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Read+0x12c k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:92 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 828058834 67 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 822306572 3958 @ 0x40c378 0x936e45 0x936a3a 0x98c8eb 0x98c7b4 0x98c485 0x98bc25 0x98a005 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x936e44 google.golang.org/grpc/internal/transport.newHTTP2Client.func5+0x44 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:391 # 0x936a39 google.golang.org/grpc/internal/transport.newHTTP2Client+0x2399 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:454 # 0x98c8ea google.golang.org/grpc/internal/transport.NewClientTransport+0x38a google.golang.org/grpc@v1.51.0/internal/transport/transport.go:587 # 0x98c7b3 google.golang.org/grpc.(*addrConn).createTransport+0x253 google.golang.org/grpc@v1.51.0/clientconn.go:1269 # 0x98c484 google.golang.org/grpc.(*addrConn).tryAllAddrs+0x4c4 google.golang.org/grpc@v1.51.0/clientconn.go:1213 # 0x98bc24 google.golang.org/grpc.(*addrConn).resetTransport+0x1c4 google.golang.org/grpc@v1.51.0/clientconn.go:1148 # 0x98a004 google.golang.org/grpc.(*addrConn).connect+0x104 google.golang.org/grpc@v1.51.0/clientconn.go:803 809455732 98 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23ea510 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23ea50f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x10f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:264 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 710748702 98 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e94a2 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e94a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x121 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:152 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 679029912 98 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23ea510 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23ea50f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x10f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:264 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 581434208 2 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x107d648 0x107d3ce 0x22cd274 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x107d3cd k8s.io/client-go/tools/cache.WaitForNamedCacheSync+0x12d k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:313 # 0x22cd273 k8s.io/apiserver/pkg/server/dynamiccertificates.(*ConfigMapCAController).Run+0x2f3 k8s.io/apiserver@v0.27.4/pkg/server/dynamiccertificates/configmap_cafile_content.go:209 556701434 67 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 541060350 67 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 535593400 2 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1895249 0x1895be6 0x1ab1d34 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1895248 k8s.io/client-go/kubernetes/typed/core/v1.(*events).PatchWithEventNamespace+0x188 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:99 # 0x1895be5 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Patch+0x25 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:168 # 0x1ab1d33 k8s.io/client-go/tools/record.recordEvent+0x93 k8s.io/client-go@v0.27.4/tools/record/event.go:255 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 436538544 1 @ 0x451705 0x101adda 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x1b27465 0x1b2710b 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x101add9 golang.org/x/time/rate.(*Limiter).WaitN+0x3b9 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:256 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x1b27464 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reportDegraded+0x1e4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:225 # 0x1b2710a github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x6a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:202 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 410747654 13 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c6b8e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c6b8d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x4cd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1084 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 394196334 77 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1897851 0x1a46f27 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1897850 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).Watch+0x130 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:108 # 0x1a46f26 k8s.io/client-go/informers/core/v1.NewFilteredNamespaceInformer.func2+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/namespace.go:70 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 382298832 2 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 381498682 77 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x18994d1 0x1a47587 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x18994d0 k8s.io/client-go/kubernetes/typed/core/v1.(*nodes).Watch+0x130 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/node.go:109 # 0x1a47586 k8s.io/client-go/informers/core/v1.NewFilteredNodeInformer.func2+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/node.go:70 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 365518794 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x19b2bf4 0x1f0c690 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2bf3 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:139 # 0x1f0c68f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0x4af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:45 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 358681824 12 @ 0x40b61d 0x1080a08 0x10809b5 0x107fe2c 0x10702ea 0x107fa96 0x107301e 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x1080a07 k8s.io/client-go/tools/cache.(*processorListener).add+0x1c7 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x10809b4 k8s.io/client-go/tools/cache.(*sharedProcessor).distribute+0x174 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:782 # 0x107fe2b k8s.io/client-go/tools/cache.(*sharedIndexInformer).OnUpdate+0x16b k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:670 # 0x10702e9 k8s.io/client-go/tools/cache.processDeltas+0x249 k8s.io/client-go@v0.27.4/tools/cache/controller.go:457 # 0x107fa95 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HandleDeltas+0x175 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:638 # 0x107301d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x5bd k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:603 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 353183078 1 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 323832528 10448 @ 0x451705 0x90356e 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90356d golang.org/x/net/http2.(*serverConn).readFrames+0x18d golang.org/x/net@v0.12.0/http2/server.go:824 320410340 664 @ 0x40b61d 0x10809af 0x108095c 0x107fe2c 0x10702ea 0x107fa96 0x107301e 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x10809ae k8s.io/client-go/tools/cache.(*processorListener).add+0x16e k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x108095b k8s.io/client-go/tools/cache.(*sharedProcessor).distribute+0x11b k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:785 # 0x107fe2b k8s.io/client-go/tools/cache.(*sharedIndexInformer).OnUpdate+0x16b k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:670 # 0x10702e9 k8s.io/client-go/tools/cache.processDeltas+0x249 k8s.io/client-go@v0.27.4/tools/cache/controller.go:457 # 0x107fa95 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HandleDeltas+0x175 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:638 # 0x107301d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x5bd k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:603 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 317798106 2 @ 0x6019c5 0x60197a 0x601238 0x603e25 0x603ddf 0x1ab2498 0x1ab2c02 0x475761 # 0x6019c4 sync.(*Mutex).Lock+0xc4 sync/mutex.go:90 # 0x601979 k8s.io/klog/v2.(*loggingT).output+0x79 k8s.io/klog/v2@v2.100.1/klog.go:858 # 0x601237 k8s.io/klog/v2.(*loggingT).printfDepth+0x1f7 k8s.io/klog/v2@v2.100.1/klog.go:737 # 0x603e24 k8s.io/klog/v2.(*loggingT).printf+0x64 k8s.io/klog/v2@v2.100.1/klog.go:718 # 0x603dde k8s.io/klog/v2.Infof+0x1e k8s.io/klog/v2@v2.100.1/klog.go:1494 # 0x1ab2497 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging.func1+0x177 k8s.io/client-go@v0.27.4/tools/record/event.go:298 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 315391262 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23f02b3 0x23ecb8e 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23f02b2 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).getAllVotingMembers+0x32 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:455 # 0x23ecb8d github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0xcd github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:148 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 310587924 13 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1894cec 0x1895b22 0x1ab1e0a 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1894ceb k8s.io/client-go/kubernetes/typed/core/v1.(*events).CreateWithEventNamespace+0x10b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:59 # 0x1895b21 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Create+0x21 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:160 # 0x1ab1e09 k8s.io/client-go/tools/record.recordEvent+0x169 k8s.io/client-go@v0.27.4/tools/record/event.go:261 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 310377586 1 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x107d648 0x107d3ce 0x22bbff4 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x107d3cd k8s.io/client-go/tools/cache.WaitForNamedCacheSync+0x12d k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:313 # 0x22bbff3 k8s.io/apiserver/pkg/authentication/request/headerrequest.(*RequestHeaderAuthRequestController).Run+0x2d3 k8s.io/apiserver@v0.27.4/pkg/authentication/request/headerrequest/requestheader_controller.go:175 309176764 1117 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 293268122 4439 @ 0x4ebf4d 0x553438 0x566765 0x7bfbad 0x936dbf 0x475761 # 0x4ebf4c internal/poll.(*FD).Close+0x6c internal/poll/fd_unix.go:116 # 0x553437 net.(*netFD).Close+0x37 net/fd_posix.go:37 # 0x566764 net.(*conn).Close+0x44 net/net.go:207 # 0x7bfbac crypto/tls.(*Conn).Close+0xcc crypto/tls/conn.go:1375 # 0x936dbe google.golang.org/grpc/internal/transport.newHTTP2Client.func6+0x11e google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:450 291768756 1 @ 0x451705 0xc08a4b 0xc0783a 0xc07629 0xc0759f 0x107d648 0x1b255da 0x1b2587d 0x1f95012 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xc08a4a k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0xea k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:205 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x1b255d9 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x119 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:69 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 # 0x1f95011 github.com/openshift/library-go/pkg/operator/status.(*StatusSyncer).Run+0x231 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:102 283761866 6 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23b6ca5 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23b6ca4 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x444 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 283364422 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23f02b3 0x23ecb8e 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23f02b2 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).getAllVotingMembers+0x32 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:455 # 0x23ecb8d github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0xcd github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:148 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 269835376 790 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 262984654 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23e1fae 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 257486438 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23e1fae 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 246835972 1249 @ 0x451705 0x905645 0x90d150 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x6553e8 0x6553bd 0x6564f9 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905644 golang.org/x/net/http2.(*serverConn).writeDataFromHandler+0x204 golang.org/x/net@v0.12.0/http2/server.go:1104 # 0x90d14f golang.org/x/net/http2.(*responseWriterState).writeChunk+0x58f golang.org/x/net@v0.12.0/http2/server.go:2638 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x6553e7 compress/flate.(*huffmanBitWriter).write+0xe7 compress/flate/huffman_bit_writer.go:136 # 0x6553bc compress/flate.(*huffmanBitWriter).writeCode+0xbc compress/flate/huffman_bit_writer.go:347 # 0x6564f8 compress/flate.(*huffmanBitWriter).writeTokens+0xb8 compress/flate/huffman_bit_writer.go:580 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 245426876 12 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23ccfde 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23ccfdd github.com/openshift/library-go/pkg/operator/staticpod/controller/node.NodeController.sync+0xd5d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/node/node_controller.go:118 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 244737366 1318 @ 0x451705 0x905645 0x90d150 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x654aa6 0x654a7b 0x65667d 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905644 golang.org/x/net/http2.(*serverConn).writeDataFromHandler+0x204 golang.org/x/net@v0.12.0/http2/server.go:1104 # 0x90d14f golang.org/x/net/http2.(*responseWriterState).writeChunk+0x58f golang.org/x/net@v0.12.0/http2/server.go:2638 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x654aa5 compress/flate.(*huffmanBitWriter).write+0xe5 compress/flate/huffman_bit_writer.go:136 # 0x654a7a compress/flate.(*huffmanBitWriter).writeBits+0xba compress/flate/huffman_bit_writer.go:159 # 0x65667c compress/flate.(*huffmanBitWriter).writeTokens+0x23c compress/flate/huffman_bit_writer.go:599 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 244126422 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x2013ac6 0x201af45 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x2013ac5 github.com/openshift/client-go/machine/clientset/versioned/typed/machine/v1beta1.(*machines).Watch+0x145 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/machine/clientset/versioned/typed/machine/v1beta1/machine.go:98 # 0x201af44 github.com/openshift/client-go/machine/informers/externalversions/machine/v1beta1.NewFilteredMachineInformer.func2+0x144 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/machine/informers/externalversions/machine/v1beta1/machine.go:55 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 235797200 9 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189efdc 0x1f00db0 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189efdb k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:127 # 0x1f00daf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0x5ef github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:195 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 234943318 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x18924c6 0x1a44f45 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x18924c5 k8s.io/client-go/kubernetes/typed/core/v1.(*endpoints).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/endpoints.go:112 # 0x1a44f44 k8s.io/client-go/informers/core/v1.NewFilteredEndpointsInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/endpoints.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 228310912 217 @ 0x451705 0x91674c 0x9165be 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91674b golang.org/x/net/http2.(*clientStream).writeRequest+0x14b golang.org/x/net@v0.12.0/http2/transport.go:1348 # 0x9165bd golang.org/x/net/http2.(*clientStream).doRequest+0x1d golang.org/x/net@v0.12.0/http2/transport.go:1323 228144280 4 @ 0x481716 0x4816f5 0x1070e25 0x107b64e 0x107862e 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070e24 k8s.io/client-go/tools/cache.(*DeltaFIFO).Update+0x44 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:321 # 0x107b64d k8s.io/client-go/tools/cache.watchHandler+0x90d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:749 # 0x107862d k8s.io/client-go/tools/cache.(*Reflector).watch+0x54d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:431 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 226631694 12 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189001c 0x1f01b30 0x1f03d25 0x1f03d26 0x1f0397a 0x23b61b6 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189001b k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:123 # 0x1f01b2f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x52f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:269 # 0x1f03d24 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x1f03d25 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialConfigMap+0x325 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:492 # 0x1f03979 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncConfigMap+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:455 # 0x23b61b5 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x515 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:236 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 224437346 3 @ 0x451705 0xb9b7c5 0xb99268 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xb9b7c4 go.etcd.io/etcd/client/v3.waitRetryBackoff+0xa4 go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:311 # 0xb99267 go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x3a7 go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:50 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 212899702 545 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 212471614 37 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x18aa6c6 0x1a4b3e5 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x18aa6c5 k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:115 # 0x1a4b3e4 k8s.io/client-go/informers/core/v1.NewFilteredServiceAccountInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/serviceaccount.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 212065584 37 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1be823a 0x2262ebf 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1be8239 k8s.io/client-go/dynamic.(*dynamicResourceClient).Watch+0x399 k8s.io/client-go@v0.27.4/dynamic/simple.go:329 # 0x2262ebe k8s.io/client-go/dynamic/dynamicinformer.NewFilteredDynamicInformer.func2+0x17e k8s.io/client-go@v0.27.4/dynamic/dynamicinformer/informer.go:162 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 207059686 37 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x189ee26 0x1a489e5 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x189ee25 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:116 # 0x1a489e4 k8s.io/client-go/informers/core/v1.NewFilteredPodInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/pod.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 204130894 946 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 203615838 3 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1894cec 0x1895b22 0x1ab1e0a 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1894ceb k8s.io/client-go/kubernetes/typed/core/v1.(*events).CreateWithEventNamespace+0x10b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:59 # 0x1895b21 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Create+0x21 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:160 # 0x1ab1e09 k8s.io/client-go/tools/record.recordEvent+0x169 k8s.io/client-go@v0.27.4/tools/record/event.go:261 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 203069682 1 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff726d 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 201944596 806 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 199822624 36 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1f51531 0x1f7e806 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1f51530 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterOperators).Watch+0x130 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusteroperator.go:93 # 0x1f7e805 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredClusterOperatorInformer.func2+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/clusteroperator.go:54 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 198536614 7 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c0e89 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c0e88 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0x1728 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:501 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 198205746 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1f4c631 0x1f7d4e6 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1f4c630 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*aPIServers).Watch+0x130 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/apiserver.go:93 # 0x1f7d4e5 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredAPIServerInformer.func2+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/apiserver.go:54 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 197047938 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x19b27e6 0x1a66865 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x19b27e5 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:114 # 0x1a66864 k8s.io/client-go/informers/policy/v1.NewFilteredPodDisruptionBudgetInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/policy/v1/poddisruptionbudget.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 190874526 552 @ 0x917d32 0x917d1d 0x9165ce 0x475761 # 0x917d31 sync.(*Mutex).Lock+0x3d1 sync/mutex.go:90 # 0x917d1c golang.org/x/net/http2.(*clientStream).cleanupWriteRequest+0x3bc golang.org/x/net@v0.12.0/http2/transport.go:1586 # 0x9165cd golang.org/x/net/http2.(*clientStream).doRequest+0x2d golang.org/x/net@v0.12.0/http2/transport.go:1324 189142372 39 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x19c56b1 0x1a6ae27 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x19c56b0 k8s.io/client-go/kubernetes/typed/rbac/v1.(*clusterRoleBindings).Watch+0x130 k8s.io/client-go@v0.27.4/kubernetes/typed/rbac/v1/clusterrolebinding.go:107 # 0x1a6ae26 k8s.io/client-go/informers/rbac/v1.NewFilteredClusterRoleBindingInformer.func2+0x126 k8s.io/client-go@v0.27.4/informers/rbac/v1/clusterrolebinding.go:70 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 187275676 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1f65a11 0x1f83b49 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1f65a10 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*networks).Watch+0x130 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/network.go:93 # 0x1f83b48 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredNetworkInformer.func2+0x128 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/network.go:54 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 183651932 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1929946 0x1a3dce5 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1929945 k8s.io/client-go/kubernetes/typed/batch/v1.(*jobs).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/batch/v1/job.go:114 # 0x1a3dce4 k8s.io/client-go/informers/batch/v1.NewFilteredJobInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/batch/v1/job.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 180718026 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23ecb2a 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23ecb29 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:135 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 178144542 650 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 176260206 16 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 174960574 162 @ 0x481716 0x4816f5 0x1070b5a 0x106f662 0x107e822 0x1ff7fe7 0x1ff3805 0x1ff8bda 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070b59 k8s.io/client-go/tools/cache.(*DeltaFIFO).HasSynced+0x39 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:301 # 0x106f661 k8s.io/client-go/tools/cache.(*controller).HasSynced+0x21 k8s.io/client-go@v0.27.4/tools/cache/controller.go:169 # 0x107e821 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0xa1 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:519 # 0x1ff7fe6 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x86 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:485 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 171066598 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x18a8806 0x1a4ace5 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x18a8805 k8s.io/client-go/kubernetes/typed/core/v1.(*services).Watch+0x145 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:113 # 0x1a4ace4 k8s.io/client-go/informers/core/v1.NewFilteredServiceInformer.func2+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/service.go:71 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 170277990 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 169183684 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1890274 0x1f02630 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 164011874 40 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1f620d1 0x1f82166 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1f620d0 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*infrastructures).Watch+0x130 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/infrastructure.go:93 # 0x1f82165 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredInfrastructureInformer.func2+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/infrastructure.go:54 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 159504022 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 159493316 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x2060911 0x208c306 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x2060910 github.com/openshift/client-go/operator/clientset/versioned/typed/operator/v1.(*etcds).Watch+0x130 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/operator/clientset/versioned/typed/operator/v1/etcd.go:93 # 0x208c305 github.com/openshift/client-go/operator/informers/externalversions/operator/v1.NewFilteredEtcdInformer.func2+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/operator/informers/externalversions/operator/v1/etcd.go:54 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 159194394 37 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1f59891 0x1f80186 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1f59890 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*featureGates).Watch+0x130 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/featuregate.go:93 # 0x1f80185 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredFeatureGateInformer.func2+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/featuregate.go:54 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 158773472 928 @ 0x451705 0x905645 0x90d150 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x6553e8 0x6553bd 0x65654e 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905644 golang.org/x/net/http2.(*serverConn).writeDataFromHandler+0x204 golang.org/x/net@v0.12.0/http2/server.go:1104 # 0x90d14f golang.org/x/net/http2.(*responseWriterState).writeChunk+0x58f golang.org/x/net@v0.12.0/http2/server.go:2638 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x6553e7 compress/flate.(*huffmanBitWriter).write+0xe7 compress/flate/huffman_bit_writer.go:136 # 0x6553bc compress/flate.(*huffmanBitWriter).writeCode+0xbc compress/flate/huffman_bit_writer.go:347 # 0x65654d compress/flate.(*huffmanBitWriter).writeTokens+0x10d compress/flate/huffman_bit_writer.go:586 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 158530916 38 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x1f531d1 0x1f7ee66 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x1f531d0 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterVersions).Watch+0x130 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusterversion.go:93 # 0x1f7ee65 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredClusterVersionInformer.func2+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/clusterversion.go:54 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 156818534 25 @ 0xbd49b2 0xbd4992 0x1b27e23 0x1b27abe 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0xbd49b1 sync.(*Mutex).Lock+0x51 sync/mutex.go:90 # 0xbd4991 k8s.io/apimachinery/pkg/util/runtime.(*rudimentaryErrorBackoff).OnError+0x31 k8s.io/apimachinery@v0.27.4/pkg/util/runtime/runtime.go:129 # 0x1b27e22 k8s.io/apimachinery/pkg/util/runtime.HandleError+0x6c2 k8s.io/apimachinery@v0.27.4/pkg/util/runtime/runtime.go:109 # 0x1b27abd github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x35d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:268 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 152109792 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 151839168 73 @ 0x481716 0x4816f5 0x1070b5a 0x106f662 0x107e822 0x1ff7fd2 0x1ff3805 0x1ff8bda 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070b59 k8s.io/client-go/tools/cache.(*DeltaFIFO).HasSynced+0x39 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:301 # 0x106f661 k8s.io/client-go/tools/cache.(*controller).HasSynced+0x21 k8s.io/client-go@v0.27.4/tools/cache/controller.go:169 # 0x107e821 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0xa1 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:519 # 0x1ff7fd1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x71 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:482 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 150566190 16 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 149523038 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 137167134 16 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 135369736 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 132706662 8 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x193da1a 0x20b6511 0x20b937d 0x20b87e5 0xc05afe 0xc059b6 0xc058a9 0x20b86fa 0x20b817c 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193da19 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:78 # 0x20b6510 k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Get+0x90 k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:42 # 0x20b937c k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x17c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:324 # 0x20b87e4 k8s.io/client-go/tools/leaderelection.(*LeaderElector).acquire.func1+0x64 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:247 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b86f9 k8s.io/client-go/tools/leaderelection.(*LeaderElector).acquire+0x219 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:246 # 0x20b817b k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x5b k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:203 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 130885686 13 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 126653628 16 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23e1fae 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 125338978 281 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 122661820 3 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23f99a5 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23f99a4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x1c4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:47 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 121636952 2 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 121162160 190 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 113616396 666 @ 0x451705 0x905645 0x90d150 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x6553e8 0x6553bd 0x656630 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905644 golang.org/x/net/http2.(*serverConn).writeDataFromHandler+0x204 golang.org/x/net@v0.12.0/http2/server.go:1104 # 0x90d14f golang.org/x/net/http2.(*responseWriterState).writeChunk+0x58f golang.org/x/net@v0.12.0/http2/server.go:2638 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x6553e7 compress/flate.(*huffmanBitWriter).write+0xe7 compress/flate/huffman_bit_writer.go:136 # 0x6553bc compress/flate.(*huffmanBitWriter).writeCode+0xbc compress/flate/huffman_bit_writer.go:347 # 0x65662f compress/flate.(*huffmanBitWriter).writeTokens+0x1ef compress/flate/huffman_bit_writer.go:595 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 109178290 3 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1f51b7c 0x1f960b0 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f51b7b github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterOperators).UpdateStatus+0x19b github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusteroperator.go:131 # 0x1f960af github.com/openshift/library-go/pkg/operator/status.StatusSyncer.Sync+0x104f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:215 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 108761374 4 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x1b27465 0x1b2710b 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x1b27464 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reportDegraded+0x1e4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:225 # 0x1b2710a github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x6a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:202 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 107747948 4473 @ 0x40c378 0x937176 0x936a3a 0x98c8eb 0x98c7b4 0x98c485 0x98bc25 0x98a005 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x937175 google.golang.org/grpc/internal/transport.newHTTP2Client.func3+0x35 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:236 # 0x936a39 google.golang.org/grpc/internal/transport.newHTTP2Client+0x2399 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:454 # 0x98c8ea google.golang.org/grpc/internal/transport.NewClientTransport+0x38a google.golang.org/grpc@v1.51.0/internal/transport/transport.go:587 # 0x98c7b3 google.golang.org/grpc.(*addrConn).createTransport+0x253 google.golang.org/grpc@v1.51.0/clientconn.go:1269 # 0x98c484 google.golang.org/grpc.(*addrConn).tryAllAddrs+0x4c4 google.golang.org/grpc@v1.51.0/clientconn.go:1213 # 0x98bc24 google.golang.org/grpc.(*addrConn).resetTransport+0x1c4 google.golang.org/grpc@v1.51.0/clientconn.go:1148 # 0x98a004 google.golang.org/grpc.(*addrConn).connect+0x104 google.golang.org/grpc@v1.51.0/clientconn.go:803 106199352 4471 @ 0x40c378 0x555571 0x55504a 0x56b6e5 0x56b212 0x55fd18 0x56eeaf 0x56ed49 0x549592 0x548ee5 0x548553 0x547f6c 0x94fe05 0x9344b6 0x93483a 0x98c8eb 0x98c7b4 0x98c485 0x98bc25 0x98a005 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x555570 net.(*netFD).connect.func1+0x50 net/fd_unix.go:106 # 0x555049 net.(*netFD).connect+0x849 net/fd_unix.go:162 # 0x56b6e4 net.(*netFD).dial+0x3c4 net/sock_posix.go:151 # 0x56b211 net.socket+0x2d1 net/sock_posix.go:70 # 0x55fd17 net.internetSocket+0xf7 net/ipsock_posix.go:142 # 0x56eeae net.(*sysDialer).doDialTCP+0xee net/tcpsock_posix.go:74 # 0x56ed48 net.(*sysDialer).dialTCP+0x68 net/tcpsock_posix.go:64 # 0x549591 net.(*sysDialer).dialSingle+0x1f1 net/dial.go:580 # 0x548ee4 net.(*sysDialer).dialSerial+0x244 net/dial.go:548 # 0x548552 net.(*sysDialer).dialParallel+0x412 net/dial.go:449 # 0x547f6b net.(*Dialer).DialContext+0x70b net/dial.go:440 # 0x94fe04 google.golang.org/grpc/internal/transport.proxyDial+0xe4 google.golang.org/grpc@v1.51.0/internal/transport/proxy.go:125 # 0x9344b5 google.golang.org/grpc/internal/transport.dial+0x295 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:172 # 0x934839 google.golang.org/grpc/internal/transport.newHTTP2Client+0x199 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:211 # 0x98c8ea google.golang.org/grpc/internal/transport.NewClientTransport+0x38a google.golang.org/grpc@v1.51.0/internal/transport/transport.go:587 # 0x98c7b3 google.golang.org/grpc.(*addrConn).createTransport+0x253 google.golang.org/grpc@v1.51.0/clientconn.go:1269 # 0x98c484 google.golang.org/grpc.(*addrConn).tryAllAddrs+0x4c4 google.golang.org/grpc@v1.51.0/clientconn.go:1213 # 0x98bc24 google.golang.org/grpc.(*addrConn).resetTransport+0x1c4 google.golang.org/grpc@v1.51.0/clientconn.go:1148 # 0x98a004 google.golang.org/grpc.(*addrConn).connect+0x104 google.golang.org/grpc@v1.51.0/clientconn.go:803 106056750 653 @ 0x451705 0x90c10d 0x90d4b0 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x6553e8 0x6553bd 0x6564f9 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90c10c golang.org/x/net/http2.(*serverConn).writeHeaders+0x16c golang.org/x/net@v0.12.0/http2/server.go:2348 # 0x90d4af golang.org/x/net/http2.(*responseWriterState).writeChunk+0x8ef golang.org/x/net@v0.12.0/http2/server.go:2608 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x6553e7 compress/flate.(*huffmanBitWriter).write+0xe7 compress/flate/huffman_bit_writer.go:136 # 0x6553bc compress/flate.(*huffmanBitWriter).writeCode+0xbc compress/flate/huffman_bit_writer.go:347 # 0x6564f8 compress/flate.(*huffmanBitWriter).writeTokens+0xb8 compress/flate/huffman_bit_writer.go:580 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 97324092 9 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x23bde58 0x23c25f4 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23c25f3 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0x2d3 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:687 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 96406884 17 @ 0x9179ea 0x9179b3 0x9165ce 0x475761 # 0x9179e9 sync.(*Mutex).Lock+0x89 sync/mutex.go:90 # 0x9179b2 golang.org/x/net/http2.(*clientStream).cleanupWriteRequest+0x52 golang.org/x/net@v0.12.0/http2/transport.go:1538 # 0x9165cd golang.org/x/net/http2.(*clientStream).doRequest+0x2d golang.org/x/net@v0.12.0/http2/transport.go:1324 94518786 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23bfee5 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23bfee4 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0x784 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:533 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 92798094 184 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 92543718 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x2408be5 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x2408be4 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x204 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:73 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 89424484 1416 @ 0x451705 0x905645 0x90d150 0x90c907 0x57e202 0x90e03c 0x90edf9 0x90edf4 0x90beb9 0x90bc37 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905644 golang.org/x/net/http2.(*serverConn).writeDataFromHandler+0x204 golang.org/x/net@v0.12.0/http2/server.go:1104 # 0x90d14f golang.org/x/net/http2.(*responseWriterState).writeChunk+0x58f golang.org/x/net@v0.12.0/http2/server.go:2638 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x90e03b golang.org/x/net/http2.(*responseWriter).FlushError+0x3b golang.org/x/net@v0.12.0/http2/server.go:2775 # 0x90edf8 golang.org/x/net/http2.(*responseWriter).Flush+0x38 golang.org/x/net@v0.12.0/http2/server.go:2765 # 0x90edf3 golang.org/x/net/http2.(*responseWriter).handlerDone+0x33 golang.org/x/net@v0.12.0/http2/server.go:2943 # 0x90beb8 golang.org/x/net/http2.(*serverConn).runHandler.func1+0x218 golang.org/x/net@v0.12.0/http2/server.go:2312 # 0x90bc36 golang.org/x/net/http2.(*serverConn).runHandler+0x96 golang.org/x/net@v0.12.0/http2/server.go:2316 86868338 4 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189efdc 0x1f00db0 0x23c49f1 0x23c494c 0x23c03f6 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189efdb k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:127 # 0x1f00daf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0x5ef github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:195 # 0x23c49f0 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1050 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23c494b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).ensureInstallerPod+0xfab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:923 # 0x23c03f5 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xc95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:476 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 86148334 3 @ 0x47fefb 0x10240eb 0x23decfa 0x1087143 0x1081adc 0xc05afe 0xc059b6 0xc058a9 0x10819ab 0x1081968 0xc0853a 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x10240ea k8s.io/client-go/util/workqueue.(*Type).Add+0x6a k8s.io/client-go@v0.27.4/util/workqueue/queue.go:164 # 0x23decf9 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).eventHandler.func2+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:220 # 0x1087142 k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate+0x62 k8s.io/client-go@v0.27.4/tools/cache/controller.go:250 # 0x1081adb k8s.io/client-go/tools/cache.(*processorListener).run.func1+0xfb k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:971 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x10819aa k8s.io/apimachinery/pkg/util/wait.Until+0x6a k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x1081967 k8s.io/client-go/tools/cache.(*processorListener).run+0x27 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:967 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 84366196 1006 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb206a9 0xb96523 0xb8ee88 0xb8e7f0 0x1ffa1ed 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb206a8 go.etcd.io/etcd/api/v3/etcdserverpb.(*kVClient).Range+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:6460 # 0xb96522 go.etcd.io/etcd/client/v3.(*retryKVClient).Range+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:105 # 0xb8ee87 go.etcd.io/etcd/client/v3.(*kv).Do+0x1e7 go.etcd.io/etcd/client/v3@v3.5.7/kv.go:149 # 0xb8e7ef go.etcd.io/etcd/client/v3.(*kv).Get+0xcf go.etcd.io/etcd/client/v3@v3.5.7/kv.go:119 # 0x1ffa1ec github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x28c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:124 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 84010386 111 @ 0x481716 0x4816f5 0x1070b5a 0x106f662 0x107e822 0x1ff7fe7 0x1ff3805 0x1ff8bda 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070b59 k8s.io/client-go/tools/cache.(*DeltaFIFO).HasSynced+0x39 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:301 # 0x106f661 k8s.io/client-go/tools/cache.(*controller).HasSynced+0x21 k8s.io/client-go@v0.27.4/tools/cache/controller.go:169 # 0x107e821 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0xa1 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:519 # 0x1ff7fe6 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x86 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:485 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 83707244 3 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189f6c7 0x23c31b8 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189f6c6 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Delete+0x126 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:169 # 0x23c31b7 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0xe97 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:675 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 82448846 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff6c39 0x1ff710a 0x23f01f0 0x23ed0a5 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff6c38 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).HealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:368 # 0x1ff7109 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).HealthyVotingMembers+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:391 # 0x23f01ef github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).getHealthyVotingMembers+0x2f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:447 # 0x23ed0a4 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0x5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:177 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 80248240 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23ecb2a 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23ecb29 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:135 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 79029902 1 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff726d 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 77252372 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23b70fb 0x23b4c5b 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23b70fa github.com/openshift/library-go/pkg/operator/revisioncontroller.StaticPodLatestRevisionClient.UpdateLatestRevisionOperatorStatus+0x29a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/staticpod.go:36 # 0x23b4c5a github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x55a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:115 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 74567468 138 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 73935158 670 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 72432018 2 @ 0x481716 0x4816f5 0x1070b5a 0x106f662 0x107e822 0x1ff7fd2 0x1ff3805 0x1ff8bda 0x1ff726d 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070b59 k8s.io/client-go/tools/cache.(*DeltaFIFO).HasSynced+0x39 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:301 # 0x106f661 k8s.io/client-go/tools/cache.(*controller).HasSynced+0x21 k8s.io/client-go@v0.27.4/tools/cache/controller.go:169 # 0x107e821 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0xa1 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:519 # 0x1ff7fd1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x71 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:482 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 70853272 8 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 70739032 1 @ 0x107e7dd 0x107e7b2 0x1ff7fd2 0x1ff3805 0x1ff8bda 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x107e7dc sync.(*Mutex).Lock+0x5c sync/mutex.go:90 # 0x107e7b1 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0x31 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:513 # 0x1ff7fd1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x71 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:482 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 69017534 1 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1894cec 0x1895b22 0x1ab1e0a 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1894ceb k8s.io/client-go/kubernetes/typed/core/v1.(*events).CreateWithEventNamespace+0x10b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:59 # 0x1895b21 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Create+0x21 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:160 # 0x1ab1e09 k8s.io/client-go/tools/record.recordEvent+0x169 k8s.io/client-go@v0.27.4/tools/record/event.go:261 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 68914664 3 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18a733c 0x1f030b3 0x23f7534 0x23f74ef 0x23f6a49 0x23f6417 0x23f60f9 0x23f5db6 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a733b k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:123 # 0x1f030b2 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecretImproved+0x6f2 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:378 # 0x23f7533 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecret+0x813 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:87 # 0x23f74ee github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).ensureCertSecret+0x7ce github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:291 # 0x23f6a48 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).ensureCertsForNode+0x2a8 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:227 # 0x23f6416 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).ensureCerts+0x1f6 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:195 # 0x23f60f8 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).syncAllMasters+0x38 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:152 # 0x23f5db5 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0xf5 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:127 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 67306668 4206 @ 0x40c378 0x98b605 0x98b5e6 0xb88ad5 0x1ffa525 0x1ffa443 0x1ff9ebf 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x98b604 google.golang.org/grpc.(*ccBalancerWrapper).close+0x164 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:294 # 0x98b5e5 google.golang.org/grpc.(*ClientConn).Close+0x145 google.golang.org/grpc@v1.51.0/clientconn.go:1036 # 0xb88ad4 go.etcd.io/etcd/client/v3.(*Client).Close+0x74 go.etcd.io/etcd/client/v3@v3.5.7/client.go:151 # 0x1ffa524 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth.func1+0x24 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:111 # 0x1ffa442 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x4e2 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:139 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 65427926 139 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 64949612 116 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 64020478 9 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23beb5c 0x23bf85f 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23beb5b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.nodeToStartRevisionWith+0x2db github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:319 # 0x23bf85e github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xfe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:434 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 63863536 16 @ 0x47fefb 0x47e27e 0x10245e5 0x1b277cf 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x47e27d sync.(*Cond).Wait+0x9d sync/cond.go:71 # 0x10245e4 k8s.io/client-go/util/workqueue.(*Type).Get+0xa4 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:200 # 0x1b277ce github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x6e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:246 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 63300782 1 @ 0x47fefb 0x10240eb 0x23de6fa 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x10240ea k8s.io/client-go/util/workqueue.(*Type).Add+0x6a k8s.io/client-go@v0.27.4/util/workqueue/queue.go:164 # 0x23de6f9 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).Run.func1+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:184 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 62356296 1 @ 0x451705 0x240e13c 0x23aaef4 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x240e13b github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x249b github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:450 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 59762934 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23ec61f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23ec61e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x7e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:103 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 59120576 22 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 58396958 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x1b276e5 0x1b2710b 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x1b276e4 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reportDegraded+0x464 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:236 # 0x1b2710a github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x6a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:202 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 56716694 1 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 55785162 14 @ 0x107e7dd 0x107e7b2 0x1ff7fe7 0x1ff3805 0x1ff8bda 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x107e7dc sync.(*Mutex).Lock+0x5c sync/mutex.go:90 # 0x107e7b1 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0x31 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:513 # 0x1ff7fe6 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x86 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:485 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 53832914 1 @ 0x451705 0x9160d1 0x911fa5 0x911afb 0x8cf84b 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x1b21d12 0x1b2262a 0x23a921a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x911afa golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x8cf84a net/http.(*Transport).roundTrip+0x7ea net/http/transport.go:601 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1b21d11 github.com/openshift/library-go/pkg/operator/events.GetControllerReferenceForCurrentPod+0x111 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder.go:76 # 0x1b22629 github.com/openshift/library-go/pkg/operator/events.GetControllerReferenceForCurrentPod+0xa29 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder.go:61 # 0x23a9219 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x279 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:228 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 53149358 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1890274 0x1f02630 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 53063286 27 @ 0x47fefb 0x10240eb 0x1b28a24 0x1b289a6 0x1087143 0x1081adc 0xc05afe 0xc059b6 0xc058a9 0x10819ab 0x1081968 0xc0853a 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x10240ea k8s.io/client-go/util/workqueue.(*Type).Add+0x6a k8s.io/client-go@v0.27.4/util/workqueue/queue.go:164 # 0x1b28a23 github.com/openshift/library-go/pkg/controller/factory.syncContext.enqueueKeys+0x143 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:91 # 0x1b289a5 github.com/openshift/library-go/pkg/controller/factory.syncContext.eventHandler.func2+0xc5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:64 # 0x1087142 k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate+0x62 k8s.io/client-go@v0.27.4/tools/cache/controller.go:250 # 0x1081adb k8s.io/client-go/tools/cache.(*processorListener).run.func1+0xfb k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:971 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x10819aa k8s.io/apimachinery/pkg/util/wait.Until+0x6a k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x1081967 k8s.io/client-go/tools/cache.(*processorListener).run+0x27 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:967 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 51966828 5 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18a6f50 0x1a4a745 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a6f4f k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:95 # 0x1a4a744 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 51112476 326 @ 0x451705 0x905645 0x90d150 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x654aa6 0x654a7b 0x65659f 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905644 golang.org/x/net/http2.(*serverConn).writeDataFromHandler+0x204 golang.org/x/net@v0.12.0/http2/server.go:1104 # 0x90d14f golang.org/x/net/http2.(*responseWriterState).writeChunk+0x58f golang.org/x/net@v0.12.0/http2/server.go:2638 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x654aa5 compress/flate.(*huffmanBitWriter).write+0xe5 compress/flate/huffman_bit_writer.go:136 # 0x654a7a compress/flate.(*huffmanBitWriter).writeBits+0xba compress/flate/huffman_bit_writer.go:159 # 0x65659e compress/flate.(*huffmanBitWriter).writeTokens+0x15e compress/flate/huffman_bit_writer.go:590 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 49871190 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23ddefe 0x23de8ef 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23ddefd github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).sync+0x25d github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:123 # 0x23de8ee github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x10e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:204 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 49129074 18 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 49013604 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22c49 0xb973d0 0xb8d86e 0x1ff4f0c 0x23ea654 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22c48 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberPromote+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7110 # 0xb973cf go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberPromote+0x2f go.etcd.io/etcd/client/v3@v3.5.7/retry.go:183 # 0xb8d86d go.etcd.io/etcd/client/v3.(*cluster).MemberPromote+0x6d go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:136 # 0x1ff4f0b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberPromote+0x16b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:211 # 0x23ea653 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x253 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:283 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 48694018 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23dde18 0x23de8ef 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23dde17 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).sync+0x177 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:111 # 0x23de8ee github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x10e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:204 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 45017202 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18a733c 0x1f030b3 0x1f048bc 0x1f048bd 0x1f0437a 0x23b6465 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a733b k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:123 # 0x1f030b2 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecretImproved+0x6f2 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:378 # 0x1f048bb github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecret+0x4bb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:87 # 0x1f048bc github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialSecret+0x4bc github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:571 # 0x1f04379 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncSecret+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:516 # 0x23b6464 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:245 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 44473834 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f51319 0x1f7e966 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f51318 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterOperators).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusteroperator.go:77 # 0x1f7e965 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredClusterOperatorInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/clusteroperator.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 43288000 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23ec61f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23ec61e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x7e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:103 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 42247526 21 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1f51b7c 0x1f960b0 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f51b7b github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterOperators).UpdateStatus+0x19b github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusteroperator.go:131 # 0x1f960af github.com/openshift/library-go/pkg/operator/status.StatusSyncer.Sync+0x104f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:215 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 41945226 125 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 41220824 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1890274 0x1f02630 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 40471160 12 @ 0x107e7dd 0x107e7b2 0x1ff7fe7 0x1ff3805 0x1ff8bda 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x107e7dc sync.(*Mutex).Lock+0x5c sync/mutex.go:90 # 0x107e7b1 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0x31 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:513 # 0x1ff7fe6 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x86 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:485 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 40420080 1 @ 0x451705 0x9882f7 0x986c49 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9882f6 google.golang.org/grpc.(*ClientConn).WaitForStateChange+0xb6 google.golang.org/grpc@v1.51.0/clientconn.go:515 # 0x986c48 google.golang.org/grpc.DialContext+0x1808 google.golang.org/grpc@v1.51.0/clientconn.go:321 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 40051492 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be7c1b 0x226305f 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be7c1a k8s.io/client-go/dynamic.(*dynamicResourceClient).List+0x3da k8s.io/client-go@v0.27.4/dynamic/simple.go:299 # 0x226305e k8s.io/client-go/dynamic/dynamicinformer.NewFilteredDynamicInformer.func1+0x17e k8s.io/client-go@v0.27.4/dynamic/dynamicinformer/informer.go:156 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 39913064 128 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff6542 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff6541 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x141 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:337 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 39833514 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff6cf7 0x1ff710a 0x23f01f0 0x23ed0a5 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff6cf6 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).HealthyMembers+0x136 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:377 # 0x1ff7109 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).HealthyVotingMembers+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:391 # 0x23f01ef github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).getHealthyVotingMembers+0x2f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:447 # 0x23ed0a4 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0x5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:177 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 38851780 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18e259a 0x1b22192 0x1b21f3c 0x1b2262a 0x23a921a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18e2599 k8s.io/client-go/kubernetes/typed/apps/v1.(*replicaSets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/apps/v1/replicaset.go:86 # 0x1b22191 github.com/openshift/library-go/pkg/operator/events.GetControllerReferenceForCurrentPod+0x591 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder.go:86 # 0x1b21f3b github.com/openshift/library-go/pkg/operator/events.GetControllerReferenceForCurrentPod+0x33b github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder.go:81 # 0x1b22629 github.com/openshift/library-go/pkg/operator/events.GetControllerReferenceForCurrentPod+0xa29 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder.go:61 # 0x23a9219 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x279 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:228 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 38600070 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1890274 0x1f02630 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 37490470 5 @ 0xbd49b2 0xbd4992 0x23dea45 0x23de857 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0xbd49b1 sync.(*Mutex).Lock+0x51 sync/mutex.go:90 # 0xbd4991 k8s.io/apimachinery/pkg/util/runtime.(*rudimentaryErrorBackoff).OnError+0x31 k8s.io/apimachinery@v0.27.4/pkg/util/runtime/runtime.go:129 # 0x23dea44 k8s.io/apimachinery/pkg/util/runtime.HandleError+0x264 k8s.io/apimachinery@v0.27.4/pkg/util/runtime/runtime.go:109 # 0x23de856 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x76 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:210 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 36668730 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1897639 0x1a47087 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897638 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:92 # 0x1a47086 k8s.io/client-go/informers/core/v1.NewFilteredNamespaceInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/namespace.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 36215826 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eed6a 0x23ec777 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eed69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToRemoveLearningMember+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:325 # 0x23ec776 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x1d6 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:122 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 36134492 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x2013890 0x201b0a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x201388f github.com/openshift/client-go/machine/clientset/versioned/typed/machine/v1beta1.(*machines).List+0x16f github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/machine/clientset/versioned/typed/machine/v1beta1/machine.go:81 # 0x201b0a4 github.com/openshift/client-go/machine/informers/externalversions/machine/v1beta1.NewFilteredMachineInformer.func1+0x144 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/machine/informers/externalversions/machine/v1beta1/machine.go:49 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 36032916 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x193da1a 0x23a4279 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x23a4165 0x23a3d55 0x23a9bf8 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193da19 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:78 # 0x23a4278 github.com/openshift/library-go/pkg/config/serving.assertAPIConnection.func1+0x98 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:91 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x23a4164 github.com/openshift/library-go/pkg/config/serving.assertAPIConnection+0x104 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:90 # 0x23a3d54 github.com/openshift/library-go/pkg/config/serving.ToServerConfig+0x974 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:73 # 0x23a9bf7 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xc57 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:272 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 35683684 75 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 35247298 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189001c 0x1f01b30 0x23b5f7f 0x23b5f3a 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189001b k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:123 # 0x1f01b2f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x52f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:269 # 0x23b5f7e github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x2de github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23b5f39 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x299 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:223 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 35039424 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x22bc424 0x239fa95 0x239f1f4 0x239ec57 0x23a38d8 0x23a9bf8 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x22bc423 k8s.io/apiserver/pkg/authentication/request/headerrequest.(*RequestHeaderAuthRequestController).RunOnce+0xa3 k8s.io/apiserver@v0.27.4/pkg/authentication/request/headerrequest/requestheader_controller.go:187 # 0x239fa94 k8s.io/apiserver/pkg/server/options.(*DynamicRequestHeaderController).RunOnce+0xb4 k8s.io/apiserver@v0.27.4/pkg/server/options/authentication_dynamic_request_header.go:71 # 0x239f1f3 k8s.io/apiserver/pkg/server/options.(*DelegatingAuthenticationOptions).createRequestHeaderConfig+0x93 k8s.io/apiserver@v0.27.4/pkg/server/options/authentication.go:417 # 0x239ec56 k8s.io/apiserver/pkg/server/options.(*DelegatingAuthenticationOptions).ApplyTo+0x656 k8s.io/apiserver@v0.27.4/pkg/server/options/authentication.go:365 # 0x23a38d7 github.com/openshift/library-go/pkg/config/serving.ToServerConfig+0x4f7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:55 # 0x23a9bf7 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xc57 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:272 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 35022956 283 @ 0x451705 0x90c10d 0x90d4b0 0x90c907 0x57e202 0x90e03c 0x90edf9 0x90edf4 0x90beb9 0x90bc37 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90c10c golang.org/x/net/http2.(*serverConn).writeHeaders+0x16c golang.org/x/net@v0.12.0/http2/server.go:2348 # 0x90d4af golang.org/x/net/http2.(*responseWriterState).writeChunk+0x8ef golang.org/x/net@v0.12.0/http2/server.go:2608 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x90e03b golang.org/x/net/http2.(*responseWriter).FlushError+0x3b golang.org/x/net@v0.12.0/http2/server.go:2775 # 0x90edf8 golang.org/x/net/http2.(*responseWriter).Flush+0x38 golang.org/x/net@v0.12.0/http2/server.go:2765 # 0x90edf3 golang.org/x/net/http2.(*responseWriter).handlerDone+0x33 golang.org/x/net@v0.12.0/http2/server.go:2943 # 0x90beb8 golang.org/x/net/http2.(*serverConn).runHandler.func1+0x218 golang.org/x/net@v0.12.0/http2/server.go:2312 # 0x90bc36 golang.org/x/net/http2.(*serverConn).runHandler+0x96 golang.org/x/net@v0.12.0/http2/server.go:2316 34532780 117 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 34278416 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1890274 0x1f02630 0x1f03d25 0x1f03d26 0x1f0397a 0x23b61b6 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x1f03d24 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x1f03d25 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialConfigMap+0x325 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:492 # 0x1f03979 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncConfigMap+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:455 # 0x23b61b5 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x515 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:236 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 33530250 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189efdc 0x1f00db0 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189efdb k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:127 # 0x1f00daf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0x5ef github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:195 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 32879552 234 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb89e96 0xb92ead 0xb93848 0x1ff600f 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb89e95 go.etcd.io/etcd/client/v3.(*Client).Dial+0x235 go.etcd.io/etcd/client/v3@v3.5.7/client.go:254 # 0xb92eac go.etcd.io/etcd/client/v3.NewMaintenance.func1+0x4c go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:81 # 0xb93847 go.etcd.io/etcd/client/v3.(*maintenance).Status+0x67 go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:180 # 0x1ff600e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x12e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:298 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 32232566 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x193da1a 0x23a4279 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x23a4165 0x23a380f 0x23a9bf8 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193da19 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:78 # 0x23a4278 github.com/openshift/library-go/pkg/config/serving.assertAPIConnection.func1+0x98 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:91 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x23a4164 github.com/openshift/library-go/pkg/config/serving.assertAPIConnection+0x104 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:90 # 0x23a380e github.com/openshift/library-go/pkg/config/serving.ToServerConfig+0x42e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:49 # 0x23a9bf7 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xc57 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:272 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 30868846 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x2013ed4 0x23fd368 0x23fb48e 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x2013ed3 github.com/openshift/client-go/machine/clientset/versioned/typed/machine/v1beta1.(*machines).Update+0x173 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/machine/clientset/versioned/typed/machine/v1beta1/machine.go:123 # 0x23fd367 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).addDeletionHookToMasterMachines+0x2a7 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:255 # 0x23fb48d github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x8d github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:93 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 29592462 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1890274 0x1f02630 0x23b5f7f 0x23b5f3a 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x23b5f7e github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x2de github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23b5f39 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x299 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:223 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 29493256 229 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 29256088 285 @ 0x451705 0x905029 0x903e7b 0x902725 0x901bbf 0x8bedb3 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905028 golang.org/x/net/http2.(*serverConn).readPreface+0x168 golang.org/x/net@v0.12.0/http2/server.go:1068 # 0x903e7a golang.org/x/net/http2.(*serverConn).serve+0x3da golang.org/x/net@v0.12.0/http2/server.go:914 # 0x902724 golang.org/x/net/http2.(*Server).ServeConn+0xb04 golang.org/x/net@v0.12.0/http2/server.go:531 # 0x901bbe golang.org/x/net/http2.ConfigureServer.func1+0xfe golang.org/x/net@v0.12.0/http2/server.go:321 # 0x8bedb2 net/http.(*conn).serve+0x1292 net/http/server.go:1903 28668498 130 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 28588144 174 @ 0x451705 0x90c10d 0x90d4b0 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x654aa6 0x654a7b 0x65667d 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90c10c golang.org/x/net/http2.(*serverConn).writeHeaders+0x16c golang.org/x/net@v0.12.0/http2/server.go:2348 # 0x90d4af golang.org/x/net/http2.(*responseWriterState).writeChunk+0x8ef golang.org/x/net@v0.12.0/http2/server.go:2608 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x654aa5 compress/flate.(*huffmanBitWriter).write+0xe5 compress/flate/huffman_bit_writer.go:136 # 0x654a7a compress/flate.(*huffmanBitWriter).writeBits+0xba compress/flate/huffman_bit_writer.go:159 # 0x65667c compress/flate.(*huffmanBitWriter).writeTokens+0x23c compress/flate/huffman_bit_writer.go:599 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 26701882 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23f9aae 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23f9aad github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x2cd github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:59 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 26148578 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23fa3e5 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23fa3e4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x844 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:132 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 25590850 227 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 25540744 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f61eb9 0x1f822c6 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f61eb8 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*infrastructures).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/infrastructure.go:77 # 0x1f822c5 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredInfrastructureInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/infrastructure.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 25390562 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18aa490 0x1a4b545 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa48f k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:98 # 0x1a4b544 k8s.io/client-go/informers/core/v1.NewFilteredServiceAccountInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/serviceaccount.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 25176726 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f52fb9 0x1f7efc6 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f52fb8 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterVersions).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusterversion.go:77 # 0x1f7efc5 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredClusterVersionInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/clusterversion.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 25143928 157 @ 0x451705 0x90c10d 0x90d4b0 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x6553e8 0x6553bd 0x65654e 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90c10c golang.org/x/net/http2.(*serverConn).writeHeaders+0x16c golang.org/x/net@v0.12.0/http2/server.go:2348 # 0x90d4af golang.org/x/net/http2.(*responseWriterState).writeChunk+0x8ef golang.org/x/net@v0.12.0/http2/server.go:2608 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x6553e7 compress/flate.(*huffmanBitWriter).write+0xe7 compress/flate/huffman_bit_writer.go:136 # 0x6553bc compress/flate.(*huffmanBitWriter).writeCode+0xbc compress/flate/huffman_bit_writer.go:347 # 0x65654d compress/flate.(*huffmanBitWriter).writeTokens+0x10d compress/flate/huffman_bit_writer.go:586 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 25041468 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18a7594 0x1f03450 0x23f61bf 0x23f6186 0x23f5db6 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a7593 k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:137 # 0x1f0344f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecretImproved+0xa8f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:430 # 0x23f61be github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecret+0xfe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:87 # 0x23f6185 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).syncAllMasters+0xc5 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:170 # 0x23f5db5 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0xf5 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:127 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 24824456 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1929710 0x1a3de45 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x192970f k8s.io/client-go/kubernetes/typed/batch/v1.(*jobs).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/batch/v1/job.go:97 # 0x1a3de44 k8s.io/client-go/informers/batch/v1.NewFilteredJobInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/batch/v1/job.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 24708550 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x2408d0e 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x2408d0d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x32d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:85 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 24049204 1 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb226c9 0xb971f0 0xb8d413 0xb8d2ee 0x1ff4970 0x23e8e6d 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb226c8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberAdd+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7074 # 0xb971ef go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberAdd+0x2f go.etcd.io/etcd/client/v3@v3.5.7/retry.go:171 # 0xb8d412 go.etcd.io/etcd/client/v3.(*cluster).memberAdd+0xd2 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:94 # 0xb8d2ed go.etcd.io/etcd/client/v3.(*cluster).MemberAddAsLearner+0x2d go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:81 # 0x1ff496f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberAddAsLearner+0x30f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:184 # 0x23e8e6c github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x1ac github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:120 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 24040560 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1892290 0x1a450a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189228f k8s.io/client-go/kubernetes/typed/core/v1.(*endpoints).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/endpoints.go:95 # 0x1a450a4 k8s.io/client-go/informers/core/v1.NewFilteredEndpointsInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/endpoints.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 23607986 35 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 23559344 1087 @ 0x451705 0x903519 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x903518 golang.org/x/net/http2.(*serverConn).readFrames+0x138 golang.org/x/net@v0.12.0/http2/server.go:819 23290130 280 @ 0x451705 0x8d99a5 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x8d99a4 net/http.(*persistConn).readLoop+0xd84 net/http/transport.go:2227 22827672 159 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 22106794 279 @ 0x40b61d 0x1080a08 0x10809b5 0x107fc73 0x1070291 0x107fa96 0x107301e 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x1a83f2b 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x1080a07 k8s.io/client-go/tools/cache.(*processorListener).add+0x1c7 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x10809b4 k8s.io/client-go/tools/cache.(*sharedProcessor).distribute+0x174 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:782 # 0x107fc72 k8s.io/client-go/tools/cache.(*sharedIndexInformer).OnAdd+0x92 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:648 # 0x1070290 k8s.io/client-go/tools/cache.processDeltas+0x1f0 k8s.io/client-go@v0.27.4/tools/cache/controller.go:462 # 0x107fa95 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HandleDeltas+0x175 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:638 # 0x107301d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x5bd k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:603 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 # 0x1a83f2a k8s.io/client-go/informers.(*sharedInformerFactory).Start.func1+0x6a k8s.io/client-go@v0.27.4/informers/factory.go:150 21951694 280 @ 0x451705 0x23fe66a 0x2404be3 0x240526a 0x240260a 0x2406372 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x23fe669 github.com/prometheus/client_golang/api.(*httpClient).Do+0x2e9 github.com/prometheus/client_golang@v1.14.0/api/client.go:145 # 0x2404be2 github.com/prometheus/client_golang/api/prometheus/v1.(*apiClientImpl).Do+0x42 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:1122 # 0x2405269 github.com/prometheus/client_golang/api/prometheus/v1.(*apiClientImpl).DoGetFallback+0x209 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:1184 # 0x2402609 github.com/prometheus/client_golang/api/prometheus/v1.(*httpAPI).Query+0x389 github.com/prometheus/client_golang@v1.14.0/api/prometheus/v1/api.go:854 # 0x2406371 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller.(*FSyncController).sync+0x151 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller/fsync_controller.go:54 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 20332574 12 @ 0x91db99 0x91db80 0x5f3dc8 0x5f39bb 0x5f35f8 0x1041745 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x5f3dc7 encoding/json.(*Decoder).refill+0x187 encoding/json/stream.go:165 # 0x5f39ba encoding/json.(*Decoder).readValue+0xba encoding/json/stream.go:140 # 0x5f35f7 encoding/json.(*Decoder).Decode+0x77 encoding/json/stream.go:63 # 0x1041744 k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read+0x1a4 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:152 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 20264542 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f61c85 0x20b4254 0x23aa070 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f61c84 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*infrastructures).Get+0x144 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/infrastructure.go:61 # 0x20b4253 github.com/openshift/library-go/pkg/config/clusterstatus.GetClusterInfraStatus+0x93 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/clusterstatus/clusterstatus.go:21 # 0x23aa06f github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x10cf github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:318 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 20213374 280 @ 0x4ebf4d 0x553438 0x566765 0x7bfbad 0x8dc330 0x8dc130 0x8d0965 0x2406787 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x4ebf4c internal/poll.(*FD).Close+0x6c internal/poll/fd_unix.go:116 # 0x553437 net.(*netFD).Close+0x37 net/fd_posix.go:37 # 0x566764 net.(*conn).Close+0x44 net/net.go:207 # 0x7bfbac crypto/tls.(*Conn).Close+0xcc crypto/tls/conn.go:1375 # 0x8dc32f net/http.(*persistConn).closeLocked+0x12f net/http/transport.go:2732 # 0x8dc12f net/http.(*persistConn).close+0xaf net/http/transport.go:2717 # 0x8d0964 net/http.(*Transport).CloseIdleConnections+0x1c4 net/http/transport.go:782 # 0x2406786 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller.(*FSyncController).sync+0x566 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller/fsync_controller.go:76 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 19967818 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x19b25b0 0x1a669c5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b25af k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:97 # 0x1a669c4 k8s.io/client-go/informers/policy/v1.NewFilteredPodDisruptionBudgetInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/policy/v1/poddisruptionbudget.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 19916954 97 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 19864784 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f59679 0x1f802e6 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f59678 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*featureGates).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/featuregate.go:77 # 0x1f802e5 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredFeatureGateInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/featuregate.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 19825858 58 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 19629498 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f657f9 0x1f83ca9 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f657f8 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*networks).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/network.go:77 # 0x1f83ca8 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredNetworkInformer.func1+0x128 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/network.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 19491288 8 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1895249 0x1895be6 0x1ab1d34 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1895248 k8s.io/client-go/kubernetes/typed/core/v1.(*events).PatchWithEventNamespace+0x188 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:99 # 0x1895be5 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Patch+0x25 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:168 # 0x1ab1d33 k8s.io/client-go/tools/record.recordEvent+0x93 k8s.io/client-go@v0.27.4/tools/record/event.go:255 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 19168838 81 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 19074860 2 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 18818448 2 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18992b9 0x1a476e7 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18992b8 k8s.io/client-go/kubernetes/typed/core/v1.(*nodes).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/node.go:93 # 0x1a476e6 k8s.io/client-go/informers/core/v1.NewFilteredNodeInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/node.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 18686916 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1892290 0x1a450a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189228f k8s.io/client-go/kubernetes/typed/core/v1.(*endpoints).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/endpoints.go:95 # 0x1a450a4 k8s.io/client-go/informers/core/v1.NewFilteredEndpointsInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/endpoints.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 18321514 118 @ 0x451705 0x90c10d 0x90d4b0 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x6553e8 0x6553bd 0x656630 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90c10c golang.org/x/net/http2.(*serverConn).writeHeaders+0x16c golang.org/x/net@v0.12.0/http2/server.go:2348 # 0x90d4af golang.org/x/net/http2.(*responseWriterState).writeChunk+0x8ef golang.org/x/net@v0.12.0/http2/server.go:2608 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x6553e7 compress/flate.(*huffmanBitWriter).write+0xe7 compress/flate/huffman_bit_writer.go:136 # 0x6553bc compress/flate.(*huffmanBitWriter).writeCode+0xbc compress/flate/huffman_bit_writer.go:347 # 0x65662f compress/flate.(*huffmanBitWriter).writeTokens+0x1ef compress/flate/huffman_bit_writer.go:595 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 17999692 26 @ 0x9174be 0x917494 0x916be8 0x9165be 0x475761 # 0x9174bd sync.(*Mutex).Lock+0x9d sync/mutex.go:90 # 0x917493 golang.org/x/net/http2.(*clientStream).encodeAndWriteHeaders+0x73 golang.org/x/net@v0.12.0/http2/transport.go:1483 # 0x916be7 golang.org/x/net/http2.(*clientStream).writeRequest+0x5e7 golang.org/x/net@v0.12.0/http2/transport.go:1405 # 0x9165bd golang.org/x/net/http2.(*clientStream).doRequest+0x1d golang.org/x/net@v0.12.0/http2/transport.go:1323 17984096 55 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23c49f1 0x23c494c 0x23c03f6 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23c49f0 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1050 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23c494b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).ensureInstallerPod+0xfab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:923 # 0x23c03f5 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xc95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:476 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 17331268 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189ebf0 0x1a48b45 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x1a48b44 k8s.io/client-go/informers/core/v1.NewFilteredPodInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/pod.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 16787768 93 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 16603470 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eed6a 0x23ec777 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eed69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToRemoveLearningMember+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:325 # 0x23ec776 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x1d6 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:122 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 16557780 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f4c419 0x1f7d646 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f4c418 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*aPIServers).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/apiserver.go:77 # 0x1f7d645 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredAPIServerInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/apiserver.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 16447782 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x19c5499 0x1a6af87 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19c5498 k8s.io/client-go/kubernetes/typed/rbac/v1.(*clusterRoleBindings).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/rbac/v1/clusterrolebinding.go:91 # 0x1a6af86 k8s.io/client-go/informers/rbac/v1.NewFilteredClusterRoleBindingInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/rbac/v1/clusterrolebinding.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 16248866 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x20606f9 0x208c466 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x20606f8 github.com/openshift/client-go/operator/clientset/versioned/typed/operator/v1.(*etcds).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/operator/clientset/versioned/typed/operator/v1/etcd.go:77 # 0x208c465 github.com/openshift/client-go/operator/informers/externalversions/operator/v1.NewFilteredEtcdInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/operator/informers/externalversions/operator/v1/etcd.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 16093052 68 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 15833184 1 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff47a7 0x23e8e6d 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff47a6 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberAddAsLearner+0x146 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:162 # 0x23e8e6c github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x1ac github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:120 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 15743788 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1890274 0x1f02630 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 15725836 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff6479 0x23ed5a2 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23ed5a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0xae1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:199 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 15358658 1082 @ 0x40b61d 0x92a36b 0x904625 0x902725 0x901bbf 0x8bedb3 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x92a36a golang.org/x/net/http2.gate.Done+0x2a golang.org/x/net@v0.12.0/http2/http2.go:216 # 0x904624 golang.org/x/net/http2.(*serverConn).serve+0xb84 golang.org/x/net@v0.12.0/http2/server.go:960 # 0x902724 golang.org/x/net/http2.(*Server).ServeConn+0xb04 golang.org/x/net@v0.12.0/http2/server.go:531 # 0x901bbe golang.org/x/net/http2.ConfigureServer.func1+0xfe golang.org/x/net@v0.12.0/http2/server.go:321 # 0x8bedb2 net/http.(*conn).serve+0x1292 net/http/server.go:1903 15325406 163 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 14689210 15 @ 0x47e26c 0x90048b 0x91d895 0x57c57f 0x57cd4c 0x659de2 0x658b05 0x65825e 0x65bf1b 0x9205e5 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18a6f50 0x1a4a745 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x57c57e bufio.(*Reader).fill+0xfe bufio/bufio.go:106 # 0x57cd4b bufio.(*Reader).ReadByte+0x2b bufio/bufio.go:265 # 0x659de1 compress/flate.(*decompressor).huffSym+0x101 compress/flate/inflate.go:719 # 0x658b04 compress/flate.(*decompressor).huffmanBlock+0x44 compress/flate/inflate.go:494 # 0x65825d compress/flate.(*decompressor).Read+0x7d compress/flate/inflate.go:347 # 0x65bf1a compress/gzip.(*Reader).Read+0xba compress/gzip/gunzip.go:252 # 0x9205e4 golang.org/x/net/http2.(*gzipReader).Read+0xe4 golang.org/x/net@v0.12.0/http2/transport.go:3072 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a6f4f k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:95 # 0x1a4a744 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 14327888 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x193e2b4 0x20b6ccb 0x20b962f 0x20b87e5 0xc05afe 0xc059b6 0xc058a9 0x20b86fa 0x20b817c 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193e2b3 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:137 # 0x20b6cca k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Update+0x28a k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:75 # 0x20b962e k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x42e k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:363 # 0x20b87e4 k8s.io/client-go/tools/leaderelection.(*LeaderElector).acquire.func1+0x64 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:247 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b86f9 k8s.io/client-go/tools/leaderelection.(*LeaderElector).acquire+0x219 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:246 # 0x20b817b k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x5b k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:203 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 14292178 154 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 14083192 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb206a9 0xb96523 0xb8ee88 0xb8e7f0 0x1ffa1b0 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb206a8 go.etcd.io/etcd/api/v3/etcdserverpb.(*kVClient).Range+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:6460 # 0xb96522 go.etcd.io/etcd/client/v3.(*retryKVClient).Range+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:105 # 0xb8ee87 go.etcd.io/etcd/client/v3.(*kv).Do+0x1e7 go.etcd.io/etcd/client/v3@v3.5.7/kv.go:149 # 0xb8e7ef go.etcd.io/etcd/client/v3.(*kv).Get+0xcf go.etcd.io/etcd/client/v3@v3.5.7/kv.go:119 # 0x1ffa1af github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x24f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:121 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 13828586 4 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1897639 0x1a47087 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897638 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:92 # 0x1a47086 k8s.io/client-go/informers/core/v1.NewFilteredNamespaceInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/namespace.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 13627110 67 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff6542 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff6541 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x141 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:337 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 13587680 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff4e0f 0x23ea654 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff4e0e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberPromote+0x6e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:189 # 0x23ea653 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x253 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:283 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 13012478 2 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff6542 0x23ed5a2 0x23ec869 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff6541 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x141 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:337 # 0x23ed5a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0xae1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:199 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 12734904 5 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23ccfde 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23ccfdd github.com/openshift/library-go/pkg/operator/staticpod/controller/node.NodeController.sync+0xd5d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/node/node_controller.go:118 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 12081356 108 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb206a9 0xb96523 0xb8ee88 0xb8e7f0 0x1ffa1ed 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb206a8 go.etcd.io/etcd/api/v3/etcdserverpb.(*kVClient).Range+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:6460 # 0xb96522 go.etcd.io/etcd/client/v3.(*retryKVClient).Range+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:105 # 0xb8ee87 go.etcd.io/etcd/client/v3.(*kv).Do+0x1e7 go.etcd.io/etcd/client/v3@v3.5.7/kv.go:149 # 0xb8e7ef go.etcd.io/etcd/client/v3.(*kv).Get+0xcf go.etcd.io/etcd/client/v3@v3.5.7/kv.go:119 # 0x1ffa1ec github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x28c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:124 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 11832826 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1f510e5 0x240c1d3 0x23aaef4 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f510e4 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterOperators).Get+0x144 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusteroperator.go:61 # 0x240c1d2 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x532 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:149 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 11758290 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x22bc424 0x239fa95 0x22d5fb7 0x238aefb 0x238ba73 0x23838ae 0x238153d 0x23aa655 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x22bc423 k8s.io/apiserver/pkg/authentication/request/headerrequest.(*RequestHeaderAuthRequestController).RunOnce+0xa3 k8s.io/apiserver@v0.27.4/pkg/authentication/request/headerrequest/requestheader_controller.go:187 # 0x239fa94 k8s.io/apiserver/pkg/server/options.(*DynamicRequestHeaderController).RunOnce+0xb4 k8s.io/apiserver@v0.27.4/pkg/server/options/authentication_dynamic_request_header.go:71 # 0x22d5fb6 k8s.io/apiserver/pkg/server/dynamiccertificates.unionCAContent.RunOnce+0xf6 k8s.io/apiserver@v0.27.4/pkg/server/dynamiccertificates/union_content.go:89 # 0x238aefa k8s.io/apiserver/pkg/server.(*SecureServingInfo).tlsConfig+0x49a k8s.io/apiserver@v0.27.4/pkg/server/secure_serving.go:110 # 0x238ba72 k8s.io/apiserver/pkg/server.(*SecureServingInfo).Serve+0x72 k8s.io/apiserver@v0.27.4/pkg/server/secure_serving.go:160 # 0x23838ad k8s.io/apiserver/pkg/server.preparedGenericAPIServer.NonBlockingRun+0x8d k8s.io/apiserver@v0.27.4/pkg/server/genericapiserver.go:714 # 0x238153c k8s.io/apiserver/pkg/server.preparedGenericAPIServer.Run+0x55c k8s.io/apiserver@v0.27.4/pkg/server/genericapiserver.go:578 # 0x23aa654 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run.func3+0x54 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:290 11723036 16 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 11477776 280 @ 0x8d8e10 0x8d8df8 0x475761 # 0x8d8e0f sync.(*Mutex).Lock+0x1ef sync/mutex.go:90 # 0x8d8df7 net/http.(*persistConn).readLoop+0x1d7 net/http/transport.go:2109 11471678 88 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 11030818 3 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 10944116 1 @ 0x47e26c 0x90048b 0x91d895 0x57c57f 0x57cd4c 0x659de2 0x658b05 0x65815c 0x65825e 0x65bf1b 0x9205e5 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189ebf0 0x1a48b45 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x57c57e bufio.(*Reader).fill+0xfe bufio/bufio.go:106 # 0x57cd4b bufio.(*Reader).ReadByte+0x2b bufio/bufio.go:265 # 0x659de1 compress/flate.(*decompressor).huffSym+0x101 compress/flate/inflate.go:719 # 0x658b04 compress/flate.(*decompressor).huffmanBlock+0x44 compress/flate/inflate.go:494 # 0x65815b compress/flate.(*decompressor).nextBlock+0x19b compress/flate/inflate.go:327 # 0x65825d compress/flate.(*decompressor).Read+0x7d compress/flate/inflate.go:347 # 0x65bf1a compress/gzip.(*Reader).Read+0xba compress/gzip/gunzip.go:252 # 0x9205e4 golang.org/x/net/http2.(*gzipReader).Read+0xe4 golang.org/x/net@v0.12.0/http2/transport.go:3072 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x1a48b44 k8s.io/client-go/informers/core/v1.NewFilteredPodInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/pod.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 10295940 280 @ 0x40c378 0x8d9cb1 0x8dc76a 0x8dc74f 0x521158 0x23fe917 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x8d9cb0 net/http.(*persistConn).readLoop.func4+0xb0 net/http/transport.go:2199 # 0x8dc769 net/http.(*bodyEOFSignal).condfn+0x209 net/http/transport.go:2823 # 0x8dc74e net/http.(*bodyEOFSignal).Read+0x1ee net/http/transport.go:2799 # 0x521157 bytes.(*Buffer).ReadFrom+0x97 bytes/buffer.go:202 # 0x23fe916 github.com/prometheus/client_golang/api.(*httpClient).Do.func2+0x76 github.com/prometheus/client_golang@v1.14.0/api/client.go:140 10270714 68 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 10100206 73 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23c23ff 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23c23fe github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0xde github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:657 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 10075080 85 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 9942980 3 @ 0x102f6f4 0x102f726 0x102c396 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x102f6f3 sync.(*RWMutex).RLock+0x93 sync/rwmutex.go:71 # 0x102f725 k8s.io/client-go/transport.(*cachingTokenSource).Token+0xc5 k8s.io/client-go@v0.27.4/transport/token_source.go:165 # 0x102c395 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x295 k8s.io/client-go@v0.27.4/transport/round_trippers.go:312 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 9940392 1 @ 0x451705 0x950c1c 0x9a631e 0x9a630b 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff46dd 0x23e8e6d 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x950c1b google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader+0x7b google.golang.org/grpc@v1.51.0/internal/transport/transport.go:328 # 0x9a631d google.golang.org/grpc/internal/transport.(*Stream).RecvCompress+0xbd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:343 # 0x9a630a google.golang.org/grpc.(*csAttempt).recvMsg+0xaa google.golang.org/grpc@v1.51.0/stream.go:1027 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff46dc github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberAddAsLearner+0x7c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:153 # 0x23e8e6c github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x1ac github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:120 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 9898034 2 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23beb5c 0x23bf85f 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23beb5b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.nodeToStartRevisionWith+0x2db github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:319 # 0x23bf85e github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xfe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:434 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 9636412 1 @ 0x451705 0x91e052 0x920669 0x1064a44 0x105f972 0x105f685 0x105fce9 0x19c5499 0x1a6af87 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x920668 golang.org/x/net/http2.(*gzipReader).Close+0x28 golang.org/x/net@v0.12.0/http2/transport.go:3076 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19c5498 k8s.io/client-go/kubernetes/typed/rbac/v1.(*clusterRoleBindings).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/rbac/v1/clusterrolebinding.go:91 # 0x1a6af86 k8s.io/client-go/informers/rbac/v1.NewFilteredClusterRoleBindingInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/rbac/v1/clusterrolebinding.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 8741152 86 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 8061860 87 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 7820670 2 @ 0x91ecbf 0x91ec93 0x91c3d1 0x91bcb2 0x91af6f 0x475761 # 0x91ecbe sync.(*Mutex).Lock+0x5e sync/mutex.go:90 # 0x91ec92 golang.org/x/net/http2.(*clientConnReadLoop).streamByID+0x32 golang.org/x/net@v0.12.0/http2/transport.go:2757 # 0x91c3d0 golang.org/x/net/http2.(*clientConnReadLoop).processHeaders+0x30 golang.org/x/net@v0.12.0/http2/transport.go:2311 # 0x91bcb1 golang.org/x/net/http2.(*clientConnReadLoop).run+0x391 golang.org/x/net@v0.12.0/http2/transport.go:2283 # 0x91af6e golang.org/x/net/http2.(*ClientConn).readLoop+0x6e golang.org/x/net@v0.12.0/http2/transport.go:2146 7496116 15 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1897405 0x1eff858 0x1f06ddf 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897404 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:76 # 0x1eff857 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyNamespaceImproved+0xb7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:92 # 0x1f06dde github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x41e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:108 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 7478586 41 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x18aa23a 0x1f00fa5 0x1f070db 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa239 k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:81 # 0x1f00fa4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceAccountImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:229 # 0x1f070da github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x71a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:126 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 7390306 2 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18a6f50 0x1a4a745 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a6f4f k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:95 # 0x1a4a744 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 7381298 40 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 6946840 84 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 6634840 1 @ 0x4809ff 0x4809da 0x48094e 0x480685 0x8f68da 0x8f6f7e 0x8f6e4c 0x90088c 0x91e305 0x91bd14 0x91af6f 0x475761 # 0x4809fe sync.(*Mutex).Lock+0x7e sync/mutex.go:90 # 0x4809d9 sync.(*Pool).pinSlow+0x59 sync/pool.go:216 # 0x48094d sync.(*Pool).pin+0x4d sync/pool.go:209 # 0x480684 sync.(*Pool).Get+0x24 sync/pool.go:131 # 0x8f68d9 golang.org/x/net/http2.getDataBufferChunk+0x59 golang.org/x/net@v0.12.0/http2/databuffer.go:47 # 0x8f6f7d golang.org/x/net/http2.(*dataBuffer).lastChunkOrAlloc+0x5d golang.org/x/net@v0.12.0/http2/databuffer.go:142 # 0x8f6e4b golang.org/x/net/http2.(*dataBuffer).Write+0x8b golang.org/x/net@v0.12.0/http2/databuffer.go:125 # 0x90088b golang.org/x/net/http2.(*pipe).Write+0x18b golang.org/x/net@v0.12.0/http2/pipe.go:94 # 0x91e304 golang.org/x/net/http2.(*clientConnReadLoop).processData+0x224 golang.org/x/net@v0.12.0/http2/transport.go:2696 # 0x91bd13 golang.org/x/net/http2.(*clientConnReadLoop).run+0x3f3 golang.org/x/net@v0.12.0/http2/transport.go:2285 # 0x91af6e golang.org/x/net/http2.(*ClientConn).readLoop+0x6e golang.org/x/net@v0.12.0/http2/transport.go:2146 6581648 32 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 6533630 2 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 6477742 3 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 6385202 9 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 6349320 9 @ 0x900b72 0x900b4e 0x917c05 0x917bdd 0x9165ce 0x475761 # 0x900b71 sync.(*Mutex).Lock+0x71 sync/mutex.go:90 # 0x900b4d golang.org/x/net/http2.(*pipe).closeWithError+0x4d golang.org/x/net@v0.12.0/http2/pipe.go:117 # 0x917c04 golang.org/x/net/http2.(*pipe).CloseWithError+0x2a4 golang.org/x/net@v0.12.0/http2/pipe.go:102 # 0x917bdc golang.org/x/net/http2.(*clientStream).cleanupWriteRequest+0x27c golang.org/x/net@v0.12.0/http2/transport.go:1580 # 0x9165cd golang.org/x/net/http2.(*clientStream).doRequest+0x2d golang.org/x/net@v0.12.0/http2/transport.go:1324 6207720 1 @ 0x9142b7 0x914297 0x8f503f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 # 0x9142b6 sync.(*Mutex).Lock+0x56 sync/mutex.go:90 # 0x914296 golang.org/x/net/http2.(*ClientConn).ReserveNewRequest+0x36 golang.org/x/net@v0.12.0/http2/transport.go:882 # 0x8f503e golang.org/x/net/http2.(*clientConnPool).getClientConn+0x29e golang.org/x/net@v0.12.0/http2/client_conn_pool.go:78 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 6192562 60 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5f46 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5f45 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x65 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:290 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 6188640 4 @ 0x102f6f4 0x102f726 0x102c396 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x102f6f3 sync.(*RWMutex).RLock+0x93 sync/rwmutex.go:71 # 0x102f725 k8s.io/client-go/transport.(*cachingTokenSource).Token+0xc5 k8s.io/client-go@v0.27.4/transport/token_source.go:165 # 0x102c395 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x295 k8s.io/client-go@v0.27.4/transport/round_trippers.go:312 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 6068886 45 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 5733522 2 @ 0x47fefb 0x47e27e 0x90048b 0x91d895 0x5f3dc8 0x5f39bb 0x5f35f8 0x1041745 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x47e27d sync.(*Cond).Wait+0x9d sync/cond.go:71 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x5f3dc7 encoding/json.(*Decoder).refill+0x187 encoding/json/stream.go:165 # 0x5f39ba encoding/json.(*Decoder).readValue+0xba encoding/json/stream.go:140 # 0x5f35f7 encoding/json.(*Decoder).Decode+0x77 encoding/json/stream.go:63 # 0x1041744 k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read+0x1a4 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:152 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 4946022 32 @ 0x451705 0x90c10d 0x90d4b0 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x654aa6 0x654a7b 0x65659f 0x655cc6 0x6511dc 0x65229f 0x653476 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x90c10c golang.org/x/net/http2.(*serverConn).writeHeaders+0x16c golang.org/x/net@v0.12.0/http2/server.go:2348 # 0x90d4af golang.org/x/net/http2.(*responseWriterState).writeChunk+0x8ef golang.org/x/net@v0.12.0/http2/server.go:2608 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x654aa5 compress/flate.(*huffmanBitWriter).write+0xe5 compress/flate/huffman_bit_writer.go:136 # 0x654a7a compress/flate.(*huffmanBitWriter).writeBits+0xba compress/flate/huffman_bit_writer.go:159 # 0x65659e compress/flate.(*huffmanBitWriter).writeTokens+0x15e compress/flate/huffman_bit_writer.go:590 # 0x655cc5 compress/flate.(*huffmanBitWriter).writeBlock+0x485 compress/flate/huffman_bit_writer.go:492 # 0x6511db compress/flate.(*compressor).writeBlock+0x9b compress/flate/deflate.go:170 # 0x65229e compress/flate.(*compressor).deflate+0x63e compress/flate/deflate.go:408 # 0x653475 compress/flate.(*compressor).close+0x75 compress/flate/deflate.go:638 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 4863534 280 @ 0x40c378 0x7c0745 0x7c0378 0x8d5230 0x8d5231 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x7c0744 crypto/tls.(*Conn).handshakeContext.func1+0x44 crypto/tls/conn.go:1461 # 0x7c0377 crypto/tls.(*Conn).handshakeContext+0x417 crypto/tls/conn.go:1507 # 0x8d522f crypto/tls.(*Conn).HandshakeContext+0x6f crypto/tls/conn.go:1434 # 0x8d5230 net/http.(*persistConn).addTLS.func2+0x70 net/http/transport.go:1544 4797330 280 @ 0x40c378 0x555571 0x55504a 0x56b6e5 0x56b212 0x55fd18 0x56eeaf 0x56ed49 0x549592 0x548ee5 0x548553 0x547f6c 0x8d2c27 0x8d5b2c 0x8d4570 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x555570 net.(*netFD).connect.func1+0x50 net/fd_unix.go:106 # 0x555049 net.(*netFD).connect+0x849 net/fd_unix.go:162 # 0x56b6e4 net.(*netFD).dial+0x3c4 net/sock_posix.go:151 # 0x56b211 net.socket+0x2d1 net/sock_posix.go:70 # 0x55fd17 net.internetSocket+0xf7 net/ipsock_posix.go:142 # 0x56eeae net.(*sysDialer).doDialTCP+0xee net/tcpsock_posix.go:74 # 0x56ed48 net.(*sysDialer).dialTCP+0x68 net/tcpsock_posix.go:64 # 0x549591 net.(*sysDialer).dialSingle+0x1f1 net/dial.go:580 # 0x548ee4 net.(*sysDialer).dialSerial+0x244 net/dial.go:548 # 0x548552 net.(*sysDialer).dialParallel+0x412 net/dial.go:449 # 0x547f6b net.(*Dialer).DialContext+0x70b net/dial.go:440 # 0x8d2c26 net/http.(*Transport).dial+0xe6 net/http/transport.go:1176 # 0x8d5b2b net/http.(*Transport).dialConn+0x82b net/http/transport.go:1614 # 0x8d456f net/http.(*Transport).dialConnFor+0xaf net/http/transport.go:1456 4552428 10 @ 0x481716 0x4816f5 0x1070b5a 0x106f662 0x107e822 0x23e36e5 0x23fb45f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070b59 k8s.io/client-go/tools/cache.(*DeltaFIFO).HasSynced+0x39 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:301 # 0x106f661 k8s.io/client-go/tools/cache.(*controller).HasSynced+0x21 k8s.io/client-go@v0.27.4/tools/cache/controller.go:169 # 0x107e821 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0xa1 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:519 # 0x23e36e4 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*MachineAPI).IsFunctional+0x24 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/machine_api.go:45 # 0x23fb45e github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4516966 1 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23ccfde 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23ccfdd github.com/openshift/library-go/pkg/operator/staticpod/controller/node.NodeController.sync+0xd5d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/node/node_controller.go:118 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4269558 11 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18aa23a 0x1f00fa5 0x1f070db 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa239 k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:81 # 0x1f00fa4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceAccountImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:229 # 0x1f070da github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x71a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:126 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4200636 1 @ 0x47fefb 0x10240eb 0x240a0ab 0x240babb 0x23de132 0x23ddce5 0x23de8ef 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x10240ea k8s.io/client-go/util/workqueue.(*Type).Add+0x6a k8s.io/client-go@v0.27.4/util/workqueue/queue.go:164 # 0x240a0aa github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.NewTargetConfigController.func1+0x6a github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:70 # 0x240baba github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).Enqueue+0x1a github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:219 # 0x23de131 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).checkEnvVars+0x1f1 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:161 # 0x23ddce4 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).sync+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:109 # 0x23de8ee github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x10e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:204 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 4184138 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x18a6f50 0x1a4a745 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a6f4f k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:95 # 0x1a4a744 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 4176540 5 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4107630 29 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e94a2 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e94a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x121 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:152 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 4091310 91 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 4047822 5 @ 0x91ecbf 0x91ec93 0x91e12b 0x91bd14 0x91af6f 0x475761 # 0x91ecbe sync.(*Mutex).Lock+0x5e sync/mutex.go:90 # 0x91ec92 golang.org/x/net/http2.(*clientConnReadLoop).streamByID+0x32 golang.org/x/net@v0.12.0/http2/transport.go:2757 # 0x91e12a golang.org/x/net/http2.(*clientConnReadLoop).processData+0x4a golang.org/x/net@v0.12.0/http2/transport.go:2621 # 0x91bd13 golang.org/x/net/http2.(*clientConnReadLoop).run+0x3f3 golang.org/x/net@v0.12.0/http2/transport.go:2285 # 0x91af6e golang.org/x/net/http2.(*ClientConn).readLoop+0x6e golang.org/x/net@v0.12.0/http2/transport.go:2146 4037804 1 @ 0x91db99 0x91db80 0x57cb5b 0x4e4f5a 0x65b846 0x65b7f7 0x65b458 0x65b13e 0x92056f 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189ebf0 0x1a48b45 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x57cb5a bufio.(*Reader).Read+0x1ba bufio/bufio.go:237 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x65b845 io.ReadFull+0xa5 io/io.go:351 # 0x65b7f6 compress/gzip.(*Reader).readHeader+0x56 compress/gzip/gunzip.go:175 # 0x65b457 compress/gzip.(*Reader).Reset+0x2b7 compress/gzip/gunzip.go:113 # 0x65b13d compress/gzip.NewReader+0x3d compress/gzip/gunzip.go:94 # 0x92056e golang.org/x/net/http2.(*gzipReader).Read+0x6e golang.org/x/net@v0.12.0/http2/transport.go:3066 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x1a48b44 k8s.io/client-go/informers/core/v1.NewFilteredPodInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/pod.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 3976706 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1897405 0x1eff858 0x1f06ddf 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897404 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:76 # 0x1eff857 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyNamespaceImproved+0xb7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:92 # 0x1f06dde github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x41e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:108 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3899116 39 @ 0x481716 0x4816f5 0x47e27e 0x1072cb6 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x1a83f2b 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x47e27d sync.(*Cond).Wait+0x9d sync/cond.go:71 # 0x1072cb5 k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x255 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:575 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 # 0x1a83f2a k8s.io/client-go/informers.(*sharedInformerFactory).Start.func1+0x6a k8s.io/client-go@v0.27.4/informers/factory.go:150 3670882 21 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3623712 80 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 3561958 3 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3545590 57 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22f09 0xb97703 0xb938b2 0x1ff600f 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22f08 go.etcd.io/etcd/api/v3/etcdserverpb.(*maintenanceClient).Status+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7323 # 0xb97702 go.etcd.io/etcd/client/v3.(*retryMaintenanceClient).Status+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:202 # 0xb938b1 go.etcd.io/etcd/client/v3.(*maintenance).Status+0xd1 go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:185 # 0x1ff600e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x12e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:298 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 3335690 50 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 3311032 15 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x19c5265 0x1f0d268 0x1f07465 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19c5264 k8s.io/client-go/kubernetes/typed/rbac/v1.(*clusterRoleBindings).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/rbac/v1/clusterrolebinding.go:75 # 0x1f0d267 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyClusterRoleBinding+0xa7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/rbac.go:60 # 0x1f07464 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0xaa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:152 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3232584 51 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3207048 25 @ 0x91db99 0x91db80 0x4e4f5a 0x104142d 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x104142c k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Read+0x12c k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:92 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 3180030 1 @ 0x451705 0x9160d1 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18a85d0 0x1a4ae45 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9160d0 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x490 golang.org/x/net@v0.12.0/http2/transport.go:1293 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a85cf k8s.io/client-go/kubernetes/typed/core/v1.(*services).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:96 # 0x1a4ae44 k8s.io/client-go/informers/core/v1.NewFilteredServiceInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/service.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 3096168 36 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3068240 22 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23ea510 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23ea50f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x10f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:264 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 3003924 11 @ 0x47e26c 0x90048b 0x91d895 0x57c57f 0x57cd4c 0x659de2 0x658b05 0x65825e 0x65bf1b 0x9205e5 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x57c57e bufio.(*Reader).fill+0xfe bufio/bufio.go:106 # 0x57cd4b bufio.(*Reader).ReadByte+0x2b bufio/bufio.go:265 # 0x659de1 compress/flate.(*decompressor).huffSym+0x101 compress/flate/inflate.go:719 # 0x658b04 compress/flate.(*decompressor).huffmanBlock+0x44 compress/flate/inflate.go:494 # 0x65825d compress/flate.(*decompressor).Read+0x7d compress/flate/inflate.go:347 # 0x65bf1a compress/gzip.(*Reader).Read+0xba compress/gzip/gunzip.go:252 # 0x9205e4 golang.org/x/net/http2.(*gzipReader).Read+0xe4 golang.org/x/net@v0.12.0/http2/transport.go:3072 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 2976462 36 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2952584 30 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2936538 129 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2935832 234 @ 0x40c378 0x98b605 0x98b5e6 0xb9317d 0xb93927 0x1ff600f 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x98b604 google.golang.org/grpc.(*ccBalancerWrapper).close+0x164 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:294 # 0x98b5e5 google.golang.org/grpc.(*ClientConn).Close+0x145 google.golang.org/grpc@v1.51.0/clientconn.go:1036 # 0xb9317c go.etcd.io/etcd/client/v3.NewMaintenance.func1.2+0x1c go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:98 # 0xb93926 go.etcd.io/etcd/client/v3.(*maintenance).Status+0x146 go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:189 # 0x1ff600e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x12e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:298 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2872314 2 @ 0x47fefb 0x10240eb 0x1b28c6e 0x1b28b3c 0x1086fdf 0x1081b28 0xc05afe 0xc059b6 0xc058a9 0x10819ab 0x1081968 0xc0853a 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x10240ea k8s.io/client-go/util/workqueue.(*Type).Add+0x6a k8s.io/client-go@v0.27.4/util/workqueue/queue.go:164 # 0x1b28c6d github.com/openshift/library-go/pkg/controller/factory.syncContext.enqueueKeys+0x18d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:91 # 0x1b28b3b github.com/openshift/library-go/pkg/controller/factory.syncContext.eventHandler.func1+0x5b github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:56 # 0x1086fde k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnAdd+0x3e k8s.io/client-go@v0.27.4/tools/cache/controller.go:243 # 0x1081b27 k8s.io/client-go/tools/cache.(*processorListener).run.func1+0x147 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:973 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x10819aa k8s.io/apimachinery/pkg/util/wait.Until+0x6a k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x1081967 k8s.io/client-go/tools/cache.(*processorListener).run+0x27 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:967 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 2712742 23 @ 0x451705 0x1ae0371 0x1b12082 0x1ae32c2 0x1af3a1e 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 0x2386ed1 0x22f866a 0x8bf60f 0x23016d4 0x8bf60f 0x22f8130 0x8bf60f 0x231e268 0x8bf60f 0x22f866a 0x8bf60f 0x2301f7c 0x8bf60f 0x22f8130 0x8bf60f 0x22f866a 0x8bf60f 0x22f8130 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x1ae0370 github.com/prometheus/client_golang/prometheus.(*Registry).Gather+0x9f0 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:523 # 0x1b12081 k8s.io/component-base/metrics.(*kubeRegistry).Gather+0x21 k8s.io/component-base@v0.27.4/metrics/registry.go:286 # 0x1ae32c1 github.com/prometheus/client_golang/prometheus.(*noTransactionGatherer).Gather+0x21 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:1070 # 0x1af3a1d github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0xfd github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:135 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 # 0x2386ed0 k8s.io/apiserver/pkg/server.director.ServeHTTP+0x630 k8s.io/apiserver@v0.27.4/pkg/server/handler.go:154 # 0x22f8669 k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1+0x1c9 k8s.io/apiserver@v0.27.4/pkg/endpoints/filterlatency/filterlatency.go:110 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x23016d3 k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1+0x4f3 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/authorization.go:64 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x22f812f k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1+0x18f k8s.io/apiserver@v0.27.4/pkg/endpoints/filterlatency/filterlatency.go:84 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231e267 k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1+0x267 k8s.io/apiserver@v0.27.4/pkg/server/filters/maxinflight.go:197 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x22f8669 k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1+0x1c9 k8s.io/apiserver@v0.27.4/pkg/endpoints/filterlatency/filterlatency.go:110 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x2301f7b k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1+0x21b k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/impersonation.go:50 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x22f812f k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1+0x18f k8s.io/apiserver@v0.27.4/pkg/endpoints/filterlatency/filterlatency.go:84 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x22f8669 k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1+0x1c9 k8s.io/apiserver@v0.27.4/pkg/endpoints/filterlatency/filterlatency.go:110 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x22f812f k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1+0x18f k8s.io/apiserver@v0.27.4/pkg/endpoints/filterlatency/filterlatency.go:84 2637316 2 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1890274 0x1f02630 0x1f03d25 0x1f03d26 0x1f0397a 0x23b61b6 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x1f03d24 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x1f03d25 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialConfigMap+0x325 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:492 # 0x1f03979 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncConfigMap+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:455 # 0x23b61b5 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x515 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:236 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2547016 55 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 2518872 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189001c 0x1f01b30 0x1f03d25 0x1f03d26 0x1f0397a 0x23b61b6 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189001b k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:123 # 0x1f01b2f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x52f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:269 # 0x1f03d24 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x1f03d25 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialConfigMap+0x325 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:492 # 0x1f03979 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncConfigMap+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:455 # 0x23b61b5 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x515 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:236 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2471362 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1f61eb9 0x1f822c6 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f61eb8 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*infrastructures).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/infrastructure.go:77 # 0x1f822c5 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredInfrastructureInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/infrastructure.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 2461460 26 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 2443040 4 @ 0x451705 0x91fd53 0x913c50 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91fd52 golang.org/x/net/http2.(*ClientConn).Ping+0x2b2 golang.org/x/net@v0.12.0/http2/transport.go:2947 # 0x913c4f golang.org/x/net/http2.(*ClientConn).healthCheck+0xaf golang.org/x/net@v0.12.0/http2/transport.go:829 2440318 5 @ 0x40c378 0x917b46 0x9165ce 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x917b45 golang.org/x/net/http2.(*clientStream).cleanupWriteRequest+0x1e5 golang.org/x/net@v0.12.0/http2/transport.go:1551 # 0x9165cd golang.org/x/net/http2.(*clientStream).doRequest+0x2d golang.org/x/net@v0.12.0/http2/transport.go:1324 2430676 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1897639 0x1a47087 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897638 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:92 # 0x1a47086 k8s.io/client-go/informers/core/v1.NewFilteredNamespaceInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/namespace.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 2420340 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23f99a5 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23f99a4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x1c4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:47 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2340780 25 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23ea510 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23ea50f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x10f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:264 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 2292126 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x20606f9 0x208c466 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x20606f8 github.com/openshift/client-go/operator/clientset/versioned/typed/operator/v1.(*etcds).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/operator/clientset/versioned/typed/operator/v1/etcd.go:77 # 0x208c465 github.com/openshift/client-go/operator/informers/externalversions/operator/v1.NewFilteredEtcdInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/operator/informers/externalversions/operator/v1/etcd.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 2258616 34 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 2248870 49 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 2246202 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1f52fb9 0x1f7efc6 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f52fb8 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*clusterVersions).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/clusterversion.go:77 # 0x1f7efc5 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredClusterVersionInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/clusterversion.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 2125712 20 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2084600 44 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2064448 25 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 2047370 1 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1949498 20 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1774884 90 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1771694 25 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e94a2 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e94a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x121 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:152 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 1724658 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x18aa490 0x1a4b545 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa48f k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:98 # 0x1a4b544 k8s.io/client-go/informers/core/v1.NewFilteredServiceAccountInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/serviceaccount.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 1594572 3 @ 0x47e26c 0x90048b 0x91d895 0x8770b3 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x193da1a 0x20b6511 0x20b937d 0x20b8f25 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x20b8c8d 0xc05afe 0xc059b6 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x8770b2 net/http.(*cancelTimerBody).Read+0x32 net/http/client.go:955 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193da19 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:78 # 0x20b6510 k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Get+0x90 k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:42 # 0x20b937c k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x17c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:324 # 0x20b8f24 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1+0x24 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:270 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x20b8c8c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1+0x10c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:269 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 1522312 14 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1513838 13 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 1356572 27 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22f09 0xb97703 0xb938b2 0x1ff600f 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22f08 go.etcd.io/etcd/api/v3/etcdserverpb.(*maintenanceClient).Status+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7323 # 0xb97702 go.etcd.io/etcd/client/v3.(*retryMaintenanceClient).Status+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:202 # 0xb938b1 go.etcd.io/etcd/client/v3.(*maintenance).Status+0xd1 go.etcd.io/etcd/client/v3@v3.5.7/maintenance.go:185 # 0x1ff600e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x12e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:298 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 1328806 14 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be75a5 0x1f0abeb 0x1f127bf 0x1f07deb 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be75a4 k8s.io/client-go/dynamic.(*dynamicResourceClient).Get+0x544 k8s.io/client-go@v0.27.4/dynamic/simple.go:280 # 0x1f0abea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceMonitor+0x20a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/monitoring.go:60 # 0x1f127be github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyKnownUnstructured+0x1be github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/unstructured.go:18 # 0x1f07dea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x142a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:212 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1281966 20 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5f46 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5f45 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x65 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:290 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 1268884 14 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 1219184 1 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 1137182 2 @ 0x6019c5 0x60197a 0x600fb1 0x23a6066 0x23a6035 0x23a6014 0x475761 # 0x6019c4 sync.(*Mutex).Lock+0xc4 sync/mutex.go:90 # 0x601979 k8s.io/klog/v2.(*loggingT).output+0x79 k8s.io/klog/v2@v2.100.1/klog.go:858 # 0x600fb0 k8s.io/klog/v2.(*loggingT).printDepth+0x1d0 k8s.io/klog/v2@v2.100.1/klog.go:714 # 0x23a6065 k8s.io/klog/v2.(*loggingT).print+0x85 k8s.io/klog/v2@v2.100.1/klog.go:695 # 0x23a6034 k8s.io/klog/v2.Info+0x54 k8s.io/klog/v2@v2.100.1/klog.go:1470 # 0x23a6013 github.com/openshift/library-go/pkg/controller/fileobserver.(*pollingObserver).Run+0x33 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/fileobserver/observer_polling.go:159 1098280 82 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1056588 12 @ 0x481716 0x4816f5 0x1070b5a 0x106f662 0x107e822 0x107d74d 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x107d648 0x1b255da 0x1b2587d 0x23b30e8 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070b59 k8s.io/client-go/tools/cache.(*DeltaFIFO).HasSynced+0x39 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:301 # 0x106f661 k8s.io/client-go/tools/cache.(*controller).HasSynced+0x21 k8s.io/client-go@v0.27.4/tools/cache/controller.go:169 # 0x107e821 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0xa1 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:519 # 0x107d74c k8s.io/client-go/tools/cache.WaitForCacheSync.func1+0x4c k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:329 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x1b255d9 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x119 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:69 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 1041122 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 1020696 28 @ 0x40b61d 0x1080a08 0x10809b5 0x107fc73 0x1070291 0x107fa96 0x107301e 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x1080a07 k8s.io/client-go/tools/cache.(*processorListener).add+0x1c7 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x10809b4 k8s.io/client-go/tools/cache.(*sharedProcessor).distribute+0x174 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:782 # 0x107fc72 k8s.io/client-go/tools/cache.(*sharedIndexInformer).OnAdd+0x92 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:648 # 0x1070290 k8s.io/client-go/tools/cache.processDeltas+0x1f0 k8s.io/client-go@v0.27.4/tools/cache/controller.go:462 # 0x107fa95 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HandleDeltas+0x175 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:638 # 0x107301d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x5bd k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:603 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 999762 11 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23c49f1 0x23c494c 0x23c03f6 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23c49f0 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1050 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23c494b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).ensureInstallerPod+0xfab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:923 # 0x23c03f5 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xc95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:476 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 976426 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb226c9 0xb971f0 0xb8d413 0xb8d2ee 0x1ff4970 0x23e8e6d 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb226c8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberAdd+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7074 # 0xb971ef go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberAdd+0x2f go.etcd.io/etcd/client/v3@v3.5.7/retry.go:171 # 0xb8d412 go.etcd.io/etcd/client/v3.(*cluster).memberAdd+0xd2 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:94 # 0xb8d2ed go.etcd.io/etcd/client/v3.(*cluster).MemberAddAsLearner+0x2d go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:81 # 0x1ff496f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberAddAsLearner+0x30f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:184 # 0x23e8e6c github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x1ac github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:120 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 963586 20 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 907736 4 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x19c5265 0x1f0d268 0x1f07465 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19c5264 k8s.io/client-go/kubernetes/typed/rbac/v1.(*clusterRoleBindings).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/rbac/v1/clusterrolebinding.go:75 # 0x1f0d267 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyClusterRoleBinding+0xa7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/rbac.go:60 # 0x1f07464 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0xaa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:152 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 886908 2 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 866972 3 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23b6ca5 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23b6ca4 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x444 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 862592 2 @ 0x481751 0x1083e9d 0x1082573 0x10702bd 0x107fa96 0x107301e 0x106f836 0xc05afe 0xc059b6 0xc058a9 0x106f565 0x106f517 0x107e102 0x1a83f2b 0x475761 # 0x481750 sync.(*RWMutex).Lock+0x70 sync/rwmutex.go:152 # 0x1083e9c k8s.io/client-go/tools/cache.(*threadSafeMap).Update+0x5c k8s.io/client-go@v0.27.4/tools/cache/thread_safe_store.go:222 # 0x1082572 k8s.io/client-go/tools/cache.(*cache).Update+0xb2 k8s.io/client-go@v0.27.4/tools/cache/store.go:165 # 0x10702bc k8s.io/client-go/tools/cache.processDeltas+0x21c k8s.io/client-go@v0.27.4/tools/cache/controller.go:454 # 0x107fa95 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HandleDeltas+0x175 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:638 # 0x107301d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop+0x5bd k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:603 # 0x106f835 k8s.io/client-go/tools/cache.(*controller).processLoop+0x35 k8s.io/client-go@v0.27.4/tools/cache/controller.go:192 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x106f564 k8s.io/apimachinery/pkg/util/wait.Until+0x384 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x106f516 k8s.io/client-go/tools/cache.(*controller).Run+0x336 k8s.io/client-go@v0.27.4/tools/cache/controller.go:163 # 0x107e101 k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run+0x541 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:503 # 0x1a83f2a k8s.io/client-go/informers.(*sharedInformerFactory).Start.func1+0x6a k8s.io/client-go@v0.27.4/informers/factory.go:150 842208 6 @ 0x451705 0xd2a5cd 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0xd2a5cc k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0x16c k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:127 826038 10 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23ea510 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23ea50f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x10f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:264 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 810686 3 @ 0x4ebf4d 0x553438 0x566765 0x7bfbdd 0x90499f 0x902725 0x901bbf 0x8bedb3 0x475761 # 0x4ebf4c internal/poll.(*FD).Close+0x6c internal/poll/fd_unix.go:116 # 0x553437 net.(*netFD).Close+0x37 net/fd_posix.go:37 # 0x566764 net.(*conn).Close+0x44 net/net.go:207 # 0x7bfbdc crypto/tls.(*Conn).Close+0xfc crypto/tls/conn.go:1365 # 0x90499e golang.org/x/net/http2.(*serverConn).serve+0xefe golang.org/x/net@v0.12.0/http2/server.go:958 # 0x902724 golang.org/x/net/http2.(*Server).ServeConn+0xb04 golang.org/x/net@v0.12.0/http2/server.go:531 # 0x901bbe golang.org/x/net/http2.ConfigureServer.func1+0xfe golang.org/x/net@v0.12.0/http2/server.go:321 # 0x8bedb2 net/http.(*conn).serve+0x1292 net/http/server.go:1903 809204 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb88e65 0xb88df4 0x1ff8f90 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb88e64 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).SetEndpoints+0x144 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:58 # 0xb88df3 go.etcd.io/etcd/client/v3.(*Client).SetEndpoints+0xd3 go.etcd.io/etcd/client/v3@v3.5.7/client.go:177 # 0x1ff8f8f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x3ef github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:89 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 797614 28 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be75a5 0x1f0abeb 0x1f127bf 0x1f07deb 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be75a4 k8s.io/client-go/dynamic.(*dynamicResourceClient).Get+0x544 k8s.io/client-go@v0.27.4/dynamic/simple.go:280 # 0x1f0abea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceMonitor+0x20a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/monitoring.go:60 # 0x1f127be github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyKnownUnstructured+0x1be github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/unstructured.go:18 # 0x1f07dea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x142a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:212 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 751378 7 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e94a2 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e94a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x121 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:152 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 716106 2 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 676176 3 @ 0x40b61d 0x55549c 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x55549b net.(*netFD).connect.func2+0x9b net/fd_unix.go:127 653524 116 @ 0x40b61d 0x107f652 0x107f689 0x107ed87 0x1b2a9ac 0x23d7f26 0x23b30e8 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x107f651 k8s.io/client-go/tools/cache.(*processorListener).add+0x671 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x107f688 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandlerWithResyncPeriod+0x6a8 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:628 # 0x107ed86 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandler+0x26 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:556 # 0x1b2a9ab github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x9cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:294 # 0x23d7f25 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Run+0xa5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:440 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 650842 2 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 649524 13 @ 0x23ddaef 0x23ddacf 0x240a193 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x23ddaee sync.(*Mutex).Lock+0x6e sync/mutex.go:90 # 0x23ddace github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).GetEnvVars+0x4e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:98 # 0x240a192 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0xd2 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:98 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 626646 2 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18992b9 0x1a476e7 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18992b8 k8s.io/client-go/kubernetes/typed/core/v1.(*nodes).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/node.go:93 # 0x1a476e6 k8s.io/client-go/informers/core/v1.NewFilteredNodeInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/node.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 616808 2 @ 0x9142b7 0x914297 0x8f503f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x9142b6 sync.(*Mutex).Lock+0x56 sync/mutex.go:90 # 0x914296 golang.org/x/net/http2.(*ClientConn).ReserveNewRequest+0x36 golang.org/x/net@v0.12.0/http2/transport.go:882 # 0x8f503e golang.org/x/net/http2.(*clientConnPool).getClientConn+0x29e golang.org/x/net@v0.12.0/http2/client_conn_pool.go:78 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 578460 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1890274 0x1f02630 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 554430 4 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23e1fae 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 547066 9 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23ea510 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23ea50f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x10f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:264 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 535644 8 @ 0x900776 0x90073e 0x91e305 0x91bd14 0x91af6f 0x475761 # 0x900775 sync.(*Mutex).Lock+0x75 sync/mutex.go:90 # 0x90073d golang.org/x/net/http2.(*pipe).Write+0x3d golang.org/x/net@v0.12.0/http2/pipe.go:85 # 0x91e304 golang.org/x/net/http2.(*clientConnReadLoop).processData+0x224 golang.org/x/net@v0.12.0/http2/transport.go:2696 # 0x91bd13 golang.org/x/net/http2.(*clientConnReadLoop).run+0x3f3 golang.org/x/net@v0.12.0/http2/transport.go:2285 # 0x91af6e golang.org/x/net/http2.(*ClientConn).readLoop+0x6e golang.org/x/net@v0.12.0/http2/transport.go:2146 535538 4 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23e1fae 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 523754 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1897639 0x1a47087 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897638 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:92 # 0x1a47086 k8s.io/client-go/informers/core/v1.NewFilteredNamespaceInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/namespace.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 523418 16 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 501022 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189efdc 0x1f00db0 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189efdb k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:127 # 0x1f00daf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0x5ef github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:195 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 492306 28 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 486480 1 @ 0x102f6f4 0x102f726 0x102c396 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x102f6f3 sync.(*RWMutex).RLock+0x93 sync/rwmutex.go:71 # 0x102f725 k8s.io/client-go/transport.(*cachingTokenSource).Token+0xc5 k8s.io/client-go@v0.27.4/transport/token_source.go:165 # 0x102c395 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x295 k8s.io/client-go@v0.27.4/transport/round_trippers.go:312 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 450724 1 @ 0x9142b7 0x914297 0x8f503f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105d897 0x105d7b8 0x19c56b1 0x1a6ae27 0x10759b5 0x1078315 0x1077bee 0x1077786 0xc05afe 0xc059b6 0x107761d 0xc082c2 0xc0853a 0x475761 # 0x9142b6 sync.(*Mutex).Lock+0x56 sync/mutex.go:90 # 0x914296 golang.org/x/net/http2.(*ClientConn).ReserveNewRequest+0x36 golang.org/x/net@v0.12.0/http2/transport.go:882 # 0x8f503e golang.org/x/net/http2.(*clientConnPool).getClientConn+0x29e golang.org/x/net@v0.12.0/http2/client_conn_pool.go:78 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105d896 net/http.(*Client).Do+0x156 net/http/client.go:582 # 0x105d7b7 k8s.io/client-go/rest.(*Request).Watch+0x77 k8s.io/client-go@v0.27.4/rest/request.go:734 # 0x19c56b0 k8s.io/client-go/kubernetes/typed/rbac/v1.(*clusterRoleBindings).Watch+0x130 k8s.io/client-go@v0.27.4/kubernetes/typed/rbac/v1/clusterrolebinding.go:107 # 0x1a6ae26 k8s.io/client-go/informers/rbac/v1.NewFilteredClusterRoleBindingInformer.func2+0x126 k8s.io/client-go@v0.27.4/informers/rbac/v1/clusterrolebinding.go:70 # 0x10759b4 k8s.io/client-go/tools/cache.(*ListWatch).Watch+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:111 # 0x1078314 k8s.io/client-go/tools/cache.(*Reflector).watch+0x234 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:416 # 0x1077bed k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch+0x34d k8s.io/client-go@v0.27.4/tools/cache/reflector.go:356 # 0x1077785 k8s.io/client-go/tools/cache.(*Reflector).Run.func1+0x25 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:289 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0x107761c k8s.io/client-go/tools/cache.(*Reflector).Run+0x17c k8s.io/client-go@v0.27.4/tools/cache/reflector.go:288 # 0xc082c1 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:55 # 0xc08539 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1+0x59 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:72 444712 7 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 435210 1 @ 0x9142b7 0x914297 0x8f503f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x9142b6 sync.(*Mutex).Lock+0x56 sync/mutex.go:90 # 0x914296 golang.org/x/net/http2.(*ClientConn).ReserveNewRequest+0x36 golang.org/x/net@v0.12.0/http2/transport.go:882 # 0x8f503e golang.org/x/net/http2.(*clientConnPool).getClientConn+0x29e golang.org/x/net@v0.12.0/http2/client_conn_pool.go:78 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 425412 9 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 405258 5 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 405038 6 @ 0x47e26c 0x90048b 0x91d895 0x8770b3 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x193e2b4 0x20b6ccb 0x20b962f 0x20b8f25 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x20b8c8d 0xc05afe 0xc059b6 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x8770b2 net/http.(*cancelTimerBody).Read+0x32 net/http/client.go:955 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193e2b3 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:137 # 0x20b6cca k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Update+0x28a k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:75 # 0x20b962e k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x42e k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:363 # 0x20b8f24 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1+0x24 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:270 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x20b8c8c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1+0x10c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:269 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 375292 3 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 366032 6 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 359714 5 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e94a2 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e94a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x121 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:152 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 358988 4 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23c25f4 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23c25f3 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0x2d3 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:687 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 323558 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb88e65 0xb88df4 0x1ff8f90 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb88e64 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).SetEndpoints+0x144 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:58 # 0xb88df3 go.etcd.io/etcd/client/v3.(*Client).SetEndpoints+0xd3 go.etcd.io/etcd/client/v3@v3.5.7/client.go:177 # 0x1ff8f8f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x3ef github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:89 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 311554 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1f657f9 0x1f83ca9 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f657f8 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*networks).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/network.go:77 # 0x1f83ca8 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredNetworkInformer.func1+0x128 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/network.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 305660 6 @ 0x900b72 0x900b4e 0x91eb9a 0x91eb28 0x91e605 0x91bd14 0x91af6f 0x475761 # 0x900b71 sync.(*Mutex).Lock+0x71 sync/mutex.go:90 # 0x900b4d golang.org/x/net/http2.(*pipe).closeWithError+0x4d golang.org/x/net@v0.12.0/http2/pipe.go:117 # 0x91eb99 golang.org/x/net/http2.(*pipe).closeWithErrorAndCode+0x119 golang.org/x/net@v0.12.0/http2/pipe.go:111 # 0x91eb27 golang.org/x/net/http2.(*clientConnReadLoop).endStream+0xa7 golang.org/x/net@v0.12.0/http2/transport.go:2746 # 0x91e604 golang.org/x/net/http2.(*clientConnReadLoop).processData+0x524 golang.org/x/net@v0.12.0/http2/transport.go:2730 # 0x91bd13 golang.org/x/net/http2.(*clientConnReadLoop).run+0x3f3 golang.org/x/net@v0.12.0/http2/transport.go:2285 # 0x91af6e golang.org/x/net/http2.(*ClientConn).readLoop+0x6e golang.org/x/net@v0.12.0/http2/transport.go:2146 290110 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 281192 9 @ 0x47fefb 0x47e27e 0x90048b 0x91d895 0x4e4f5a 0x1041388 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x47e27d sync.(*Cond).Wait+0x9d sync/cond.go:71 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x1041387 k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Read+0x87 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:76 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 280868 24 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb206a9 0xb96523 0xb8ee88 0xb8e7f0 0x1ffa1ed 0x1ff9ebf 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb206a8 go.etcd.io/etcd/api/v3/etcdserverpb.(*kVClient).Range+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:6460 # 0xb96522 go.etcd.io/etcd/client/v3.(*retryKVClient).Range+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:105 # 0xb8ee87 go.etcd.io/etcd/client/v3.(*kv).Do+0x1e7 go.etcd.io/etcd/client/v3@v3.5.7/kv.go:149 # 0xb8e7ef go.etcd.io/etcd/client/v3.(*kv).Get+0xcf go.etcd.io/etcd/client/v3@v3.5.7/kv.go:119 # 0x1ffa1ec github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x28c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:124 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 279346 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb88e65 0xb88df4 0x1ff8f90 0x1ff5b5c 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb88e64 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).SetEndpoints+0x144 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:58 # 0xb88df3 go.etcd.io/etcd/client/v3.(*Client).SetEndpoints+0xd3 go.etcd.io/etcd/client/v3@v3.5.7/client.go:177 # 0x1ff8f8f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x3ef github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:89 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 269276 1 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23bfee5 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23bfee4 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0x784 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:533 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 267514 12 @ 0x451705 0x91e052 0x1041582 0x10357e2 0x1035e42 0xd2a3aa 0xd2a90c 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1041581 k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Close+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:108 # 0x10357e1 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Close+0x21 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:110 # 0x1035e41 k8s.io/client-go/rest/watch.(*Decoder).Close+0x21 k8s.io/client-go@v0.27.4/rest/watch/decoder.go:71 # 0xd2a3a9 k8s.io/apimachinery/pkg/watch.(*StreamWatcher).Stop+0xa9 k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:95 # 0xd2a90b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0x4ab k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:125 265432 1 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 256408 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x22bc424 0x239fa95 0x239f1f4 0x239ec57 0x23a38d8 0x23a9bf8 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x22bc423 k8s.io/apiserver/pkg/authentication/request/headerrequest.(*RequestHeaderAuthRequestController).RunOnce+0xa3 k8s.io/apiserver@v0.27.4/pkg/authentication/request/headerrequest/requestheader_controller.go:187 # 0x239fa94 k8s.io/apiserver/pkg/server/options.(*DynamicRequestHeaderController).RunOnce+0xb4 k8s.io/apiserver@v0.27.4/pkg/server/options/authentication_dynamic_request_header.go:71 # 0x239f1f3 k8s.io/apiserver/pkg/server/options.(*DelegatingAuthenticationOptions).createRequestHeaderConfig+0x93 k8s.io/apiserver@v0.27.4/pkg/server/options/authentication.go:417 # 0x239ec56 k8s.io/apiserver/pkg/server/options.(*DelegatingAuthenticationOptions).ApplyTo+0x656 k8s.io/apiserver@v0.27.4/pkg/server/options/authentication.go:365 # 0x23a38d7 github.com/openshift/library-go/pkg/config/serving.ToServerConfig+0x4f7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/serving/server.go:55 # 0x23a9bf7 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xc57 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:272 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 255724 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff726d 0x23f9c02 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f9c01 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:72 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 254910 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb88e65 0xb88df4 0x1ff8f90 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb88e64 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).SetEndpoints+0x144 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:58 # 0xb88df3 go.etcd.io/etcd/client/v3.(*Client).SetEndpoints+0xd3 go.etcd.io/etcd/client/v3@v3.5.7/client.go:177 # 0x1ff8f8f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x3ef github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:89 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 254408 24 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 251406 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff726d 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 249520 16 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 236672 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb88e65 0xb88df4 0x1ff8f90 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb88e64 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).SetEndpoints+0x144 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:58 # 0xb88df3 go.etcd.io/etcd/client/v3.(*Client).SetEndpoints+0xd3 go.etcd.io/etcd/client/v3@v3.5.7/client.go:177 # 0x1ff8f8f github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x3ef github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:89 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 229888 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 226816 11 @ 0x23ddaef 0x23ddacf 0x2409126 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x23ddaee sync.(*Mutex).Lock+0x6e sync/mutex.go:90 # 0x23ddace github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).GetEnvVars+0x4e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:98 # 0x2409125 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x85 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:114 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 226046 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1929710 0x1a3de45 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x192970f k8s.io/client-go/kubernetes/typed/batch/v1.(*jobs).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/batch/v1/job.go:97 # 0x1a3de44 k8s.io/client-go/informers/batch/v1.NewFilteredJobInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/batch/v1/job.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 224750 1 @ 0x47e26c 0x90048b 0x91d895 0x57c57f 0x57cd4c 0x659de2 0x658ef5 0x65825e 0x65bf1b 0x9205e5 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x57c57e bufio.(*Reader).fill+0xfe bufio/bufio.go:106 # 0x57cd4b bufio.(*Reader).ReadByte+0x2b bufio/bufio.go:265 # 0x659de1 compress/flate.(*decompressor).huffSym+0x101 compress/flate/inflate.go:719 # 0x658ef4 compress/flate.(*decompressor).huffmanBlock+0x434 compress/flate/inflate.go:564 # 0x65825d compress/flate.(*decompressor).Read+0x7d compress/flate/inflate.go:347 # 0x65bf1a compress/gzip.(*Reader).Read+0xba compress/gzip/gunzip.go:252 # 0x9205e4 golang.org/x/net/http2.(*gzipReader).Read+0xe4 golang.org/x/net@v0.12.0/http2/transport.go:3072 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 222578 4 @ 0x91db99 0x91db80 0x57c57f 0x57cd4c 0x659de2 0x658b05 0x65825e 0x65bf1b 0x9205e5 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18a6f50 0x1a4a745 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x57c57e bufio.(*Reader).fill+0xfe bufio/bufio.go:106 # 0x57cd4b bufio.(*Reader).ReadByte+0x2b bufio/bufio.go:265 # 0x659de1 compress/flate.(*decompressor).huffSym+0x101 compress/flate/inflate.go:719 # 0x658b04 compress/flate.(*decompressor).huffmanBlock+0x44 compress/flate/inflate.go:494 # 0x65825d compress/flate.(*decompressor).Read+0x7d compress/flate/inflate.go:347 # 0x65bf1a compress/gzip.(*Reader).Read+0xba compress/gzip/gunzip.go:252 # 0x9205e4 golang.org/x/net/http2.(*gzipReader).Read+0xe4 golang.org/x/net@v0.12.0/http2/transport.go:3072 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a6f4f k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:95 # 0x1a4a744 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 216822 7 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 213586 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x19b2bf4 0x1f0c690 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2bf3 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:139 # 0x1f0c68f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0x4af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:45 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 208872 2 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 185678 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23ecb2a 0x23ec869 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23ecb29 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToScaleDown+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:135 # 0x23ec868 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x2c8 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:125 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 182772 1 @ 0x451705 0x9831f3 0x98905d 0x9922f0 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ff379f 0x1ff8d9a 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9831f2 google.golang.org/grpc.(*ccBalancerWrapper).updateClientConnState+0xd2 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:158 # 0x98905c google.golang.org/grpc.(*ClientConn).updateResolverState+0x53c google.golang.org/grpc@v1.51.0/clientconn.go:667 # 0x9922ef google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x1cf google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:102 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ff379e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func2+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:79 # 0x1ff8d99 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x1f9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:71 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 175856 4 @ 0x451705 0x91e052 0x1041862 0x10357e2 0x1035e42 0xd2a3aa 0xd2a90c 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1041861 k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Close+0x21 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:169 # 0x10357e1 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Close+0x21 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:110 # 0x1035e41 k8s.io/client-go/rest/watch.(*Decoder).Close+0x21 k8s.io/client-go@v0.27.4/rest/watch/decoder.go:71 # 0xd2a3a9 k8s.io/apimachinery/pkg/watch.(*StreamWatcher).Stop+0xa9 k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:95 # 0xd2a90b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0x4ab k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:125 162084 7 @ 0x451705 0x91e052 0x87722a 0x1064a44 0x105f972 0x105f685 0x105fce9 0x193e2b4 0x20b6ccb 0x20b962f 0x20b8f25 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x20b8c8d 0xc05afe 0xc059b6 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x877229 net/http.(*cancelTimerBody).Close+0x29 net/http/client.go:972 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193e2b3 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:137 # 0x20b6cca k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Update+0x28a k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:75 # 0x20b962e k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x42e k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:363 # 0x20b8f24 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1+0x24 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:270 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x20b8c8c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1+0x10c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:269 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 156222 3 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18a837a 0x1efffde 0x1f06ed9 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a8379 k8s.io/client-go/kubernetes/typed/core/v1.(*services).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:79 # 0x1efffdd github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceImproved+0x15d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:139 # 0x1f06ed8 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x518 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:114 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 155976 2 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 152762 2 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1897639 0x1a47087 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897638 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:92 # 0x1a47086 k8s.io/client-go/informers/core/v1.NewFilteredNamespaceInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/namespace.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 150296 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23f9aae 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23f9aad github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x2cd github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:59 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 146300 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 137764 5 @ 0x92cccd 0x92ccab 0x92d485 0x936d0a 0x475761 # 0x92cccc sync.(*Mutex).Lock+0x6c sync/mutex.go:90 # 0x92ccaa google.golang.org/grpc/internal/transport.(*controlBuffer).get+0x4a google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:382 # 0x92d484 google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x84 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:535 # 0x936d09 google.golang.org/grpc/internal/transport.newHTTP2Client.func6+0x69 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:442 135904 1 @ 0x40c378 0x8f5aab 0x9110c8 0x8d659e 0x8d4570 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x8f5aaa golang.org/x/net/http2.(*clientConnPool).addConnIfNeeded+0x30a golang.org/x/net@v0.12.0/http2/client_conn_pool.go:183 # 0x9110c7 golang.org/x/net/http2.configureTransports.func1+0x67 golang.org/x/net@v0.12.0/http2/transport.go:254 # 0x8d659d net/http.(*Transport).dialConn+0x129d net/http/transport.go:1753 # 0x8d456f net/http.(*Transport).dialConnFor+0xaf net/http/transport.go:1456 133954 9 @ 0x47fefb 0x10252c7 0x10228a2 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x10252c6 k8s.io/client-go/util/workqueue.(*Type).ShuttingDown+0x46 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:305 # 0x10228a1 k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop+0x121 k8s.io/client-go@v0.27.4/util/workqueue/delaying_queue.go:247 123108 9 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be75a5 0x1f0abeb 0x1f127bf 0x1f07deb 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be75a4 k8s.io/client-go/dynamic.(*dynamicResourceClient).Get+0x544 k8s.io/client-go@v0.27.4/dynamic/simple.go:280 # 0x1f0abea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceMonitor+0x20a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/monitoring.go:60 # 0x1f127be github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyKnownUnstructured+0x1be github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/unstructured.go:18 # 0x1f07dea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x142a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:212 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 116794 1 @ 0x451705 0x905645 0x90d150 0x90c907 0x57e202 0x57e4b0 0x90ed1a 0x90eb50 0x2308131 0x22a071d 0x228affa 0x2321bdb 0x1af2542 0x654943 0x654910 0x6557c9 0x6534a5 0x65cf87 0x65cf77 0x1af416b 0x8bf60f 0x1b0f245 0x8bf60f 0x1af4dd4 0x8bf60f 0x1af5318 0x8bf60f 0x231936d 0x2319173 0x2386ed1 0x22f866a 0x8bf60f # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x905644 golang.org/x/net/http2.(*serverConn).writeDataFromHandler+0x204 golang.org/x/net@v0.12.0/http2/server.go:1104 # 0x90d14f golang.org/x/net/http2.(*responseWriterState).writeChunk+0x58f golang.org/x/net@v0.12.0/http2/server.go:2638 # 0x90c906 golang.org/x/net/http2.chunkWriter.Write+0x26 golang.org/x/net@v0.12.0/http2/server.go:2510 # 0x57e201 bufio.(*Writer).Flush+0x61 bufio/bufio.go:628 # 0x57e4af bufio.(*Writer).Write+0xcf bufio/bufio.go:672 # 0x90ed19 golang.org/x/net/http2.(*responseWriter).write+0x119 golang.org/x/net@v0.12.0/http2/server.go:2933 # 0x90eb4f golang.org/x/net/http2.(*responseWriter).Write+0x2f golang.org/x/net@v0.12.0/http2/server.go:2907 # 0x2308130 k8s.io/apiserver/pkg/endpoints/filters.(*writeLatencyTracker).Write+0xd0 k8s.io/apiserver@v0.27.4/pkg/endpoints/filters/webhook_duration.go:79 # 0x22a071c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).Write+0x7c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.35.1/wrap.go:80 # 0x228aff9 github.com/felixge/httpsnoop.(*rw).Write+0xb9 github.com/felixge/httpsnoop@v1.0.3/wrap_generated_gteq_1.8.go:380 # 0x2321bda k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).Write+0x1fa k8s.io/apiserver@v0.27.4/pkg/server/filters/timeout.go:213 # 0x1af2541 github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).Write+0x61 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/delegator.go:74 # 0x654942 compress/flate.(*huffmanBitWriter).write+0xc2 compress/flate/huffman_bit_writer.go:136 # 0x65490f compress/flate.(*huffmanBitWriter).flush+0x8f compress/flate/huffman_bit_writer.go:128 # 0x6557c8 compress/flate.(*huffmanBitWriter).writeStoredHeader+0x48 compress/flate/huffman_bit_writer.go:409 # 0x6534a4 compress/flate.(*compressor).close+0xa4 compress/flate/deflate.go:642 # 0x65cf86 compress/flate.(*Writer).Close+0x86 compress/flate/deflate.go:730 # 0x65cf76 compress/gzip.(*Writer).Close+0x76 compress/gzip/gzip.go:242 # 0x1af416a github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1+0x84a github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:215 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1b0f244 k8s.io/component-base/metrics.HandlerWithReset.func1+0xa4 k8s.io/component-base@v0.27.4/metrics/http.go:75 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af4dd3 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1+0xd3 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:60 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x1af5317 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1+0xb7 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:146 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 # 0x231936c k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP+0x1ac k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:242 # 0x2319172 k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP+0x72 k8s.io/apiserver@v0.27.4/pkg/server/mux/pathrecorder.go:235 # 0x2386ed0 k8s.io/apiserver/pkg/server.director.ServeHTTP+0x630 k8s.io/apiserver@v0.27.4/pkg/server/handler.go:154 # 0x22f8669 k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1+0x1c9 k8s.io/apiserver@v0.27.4/pkg/endpoints/filterlatency/filterlatency.go:110 # 0x8bf60e net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2122 115090 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 112584 4 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c6b8e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c6b8d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x4cd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1084 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 111738 1 @ 0x8f4eca 0x8f4e9f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x8f4ec9 sync.(*Mutex).Lock+0x129 sync/mutex.go:90 # 0x8f4e9e golang.org/x/net/http2.(*clientConnPool).getClientConn+0xfe golang.org/x/net@v0.12.0/http2/client_conn_pool.go:76 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 107394 8 @ 0x47e26c 0x8b8015 0x8bd3ca 0x8be17a 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x8b8014 net/http.(*connReader).abortPendingRead+0xb4 net/http/server.go:722 # 0x8bd3c9 net/http.(*response).finishRequest+0x89 net/http/server.go:1654 # 0x8be179 net/http.(*conn).serve+0x659 net/http/server.go:2001 105078 1 @ 0x102f6f4 0x102f726 0x102c396 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x18aa490 0x1a4b545 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x102f6f3 sync.(*RWMutex).RLock+0x93 sync/rwmutex.go:71 # 0x102f725 k8s.io/client-go/transport.(*cachingTokenSource).Token+0xc5 k8s.io/client-go@v0.27.4/transport/token_source.go:165 # 0x102c395 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x295 k8s.io/client-go@v0.27.4/transport/round_trippers.go:312 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa48f k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:98 # 0x1a4b544 k8s.io/client-go/informers/core/v1.NewFilteredServiceAccountInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/serviceaccount.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 103854 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23b70fb 0x23b4c5b 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23b70fa github.com/openshift/library-go/pkg/operator/revisioncontroller.StaticPodLatestRevisionClient.UpdateLatestRevisionOperatorStatus+0x29a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/staticpod.go:36 # 0x23b4c5a github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x55a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:115 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 101674 2 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 100676 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff47a7 0x23e8e6d 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff47a6 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberAddAsLearner+0x146 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:162 # 0x23e8e6c github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x1ac github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:120 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 99452 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23e1fae 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 98870 2 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 97944 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e27bd 0x23e1d85 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 95896 5 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x23c23ff 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23c23fe github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0xde github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:657 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 95810 1 @ 0x102f6f4 0x102f726 0x102c396 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x102f6f3 sync.(*RWMutex).RLock+0x93 sync/rwmutex.go:71 # 0x102f725 k8s.io/client-go/transport.(*cachingTokenSource).Token+0xc5 k8s.io/client-go@v0.27.4/transport/token_source.go:165 # 0x102c395 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x295 k8s.io/client-go@v0.27.4/transport/round_trippers.go:312 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 95276 4 @ 0x92c872 0x92c848 0x93c0bf 0x940bbf 0x9417c5 0x475761 # 0x92c871 sync.(*Mutex).Lock+0x51 sync/mutex.go:90 # 0x92c847 google.golang.org/grpc/internal/transport.(*controlBuffer).executeAndPut+0x27 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:331 # 0x93c0be google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x1fe google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:919 # 0x940bbe google.golang.org/grpc/internal/transport.(*http2Client).operateHeaders+0x1ede google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:1525 # 0x9417c4 google.golang.org/grpc/internal/transport.(*http2Client).reader+0x604 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:1594 94840 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x2408d0e 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x2408d0d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x32d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:85 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 93822 2 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 92468 2 @ 0x91da45 0x91da19 0x4e4f5a 0x1041388 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x1041387 k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Read+0x87 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:76 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 84278 2 @ 0x6019c5 0x60197a 0x601238 0x1b2568d 0x1b2563f 0x1b255e9 0x1b2587d 0x475761 # 0x6019c4 sync.(*Mutex).Lock+0xc4 sync/mutex.go:90 # 0x601979 k8s.io/klog/v2.(*loggingT).output+0x79 k8s.io/klog/v2@v2.100.1/klog.go:858 # 0x601237 k8s.io/klog/v2.(*loggingT).printfDepth+0x1f7 k8s.io/klog/v2@v2.100.1/klog.go:737 # 0x1b2568c k8s.io/klog/v2.(*loggingT).printf+0x1cc k8s.io/klog/v2@v2.100.1/klog.go:718 # 0x1b2563e k8s.io/klog/v2.Infof+0x17e k8s.io/klog/v2@v2.100.1/klog.go:1494 # 0x1b255e8 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x128 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:73 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 84160 5 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 83062 2 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1897405 0x1eff858 0x1f06ddf 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1897404 k8s.io/client-go/kubernetes/typed/core/v1.(*namespaces).Get+0x144 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/namespace.go:76 # 0x1eff857 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyNamespaceImproved+0xb7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:92 # 0x1f06dde github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x41e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:108 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 81312 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x23e27bd 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x23e27bc github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x51c github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:204 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 80436 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eed6a 0x23ec777 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eed69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).attemptToRemoveLearningMember+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:325 # 0x23ec776 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.(*clusterMemberRemovalController).sync+0x1d6 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:122 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 77816 4 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23c23ff 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23c23fe github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0xde github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:657 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 77302 1 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 74932 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x1b276e5 0x1b2710b 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x1b276e4 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reportDegraded+0x464 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:236 # 0x1b2710a github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x6a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:202 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 74252 2 @ 0x6019c5 0x60197a 0x600fb1 0xbd4925 0xbd489e 0x1b27e23 0x1b27abe 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x6019c4 sync.(*Mutex).Lock+0xc4 sync/mutex.go:90 # 0x601979 k8s.io/klog/v2.(*loggingT).output+0x79 k8s.io/klog/v2@v2.100.1/klog.go:858 # 0x600fb0 k8s.io/klog/v2.(*loggingT).printDepth+0x1d0 k8s.io/klog/v2@v2.100.1/klog.go:714 # 0xbd4924 k8s.io/klog/v2.ErrorDepth+0xa4 k8s.io/klog/v2@v2.100.1/klog.go:1560 # 0xbd489d k8s.io/apimachinery/pkg/util/runtime.logError+0x1d k8s.io/apimachinery@v0.27.4/pkg/util/runtime/runtime.go:115 # 0x1b27e22 k8s.io/apimachinery/pkg/util/runtime.HandleError+0x6c2 k8s.io/apimachinery@v0.27.4/pkg/util/runtime/runtime.go:109 # 0x1b27abd github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x35d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:268 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 72710 1 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c6b8e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c6b8d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x4cd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1084 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 72530 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b23fb7 0x23a93a6 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b23fb6 github.com/openshift/library-go/pkg/operator/events.NewKubeRecorderWithOptions+0x176 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:26 # 0x23a93a5 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x405 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:233 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 72198 2 @ 0x102f6f4 0x102f726 0x102c396 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x102f6f3 sync.(*RWMutex).RLock+0x93 sync/rwmutex.go:71 # 0x102f725 k8s.io/client-go/transport.(*cachingTokenSource).Token+0xc5 k8s.io/client-go@v0.27.4/transport/token_source.go:165 # 0x102c395 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x295 k8s.io/client-go@v0.27.4/transport/round_trippers.go:312 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 68054 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18a733c 0x1f030b3 0x1f048bc 0x1f048bd 0x1f0437a 0x23b6465 0x23b48ef 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a733b k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:123 # 0x1f030b2 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecretImproved+0x6f2 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:378 # 0x1f048bb github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecret+0x4bb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:87 # 0x1f048bc github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialSecret+0x4bc github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:571 # 0x1f04379 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncSecret+0xd9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:516 # 0x23b6464 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createNewRevision+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:245 # 0x23b48ee github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x1ee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:97 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 66946 12 @ 0x40b61d 0x107f652 0x107f689 0x107ed87 0x1b2a9ac 0x23bdcd8 0x23b30e8 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x107f651 k8s.io/client-go/tools/cache.(*processorListener).add+0x671 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x107f688 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandlerWithResyncPeriod+0x6a8 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:628 # 0x107ed86 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandler+0x26 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:556 # 0x1b2a9ab github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x9cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:294 # 0x23bdcd7 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).Run+0xd7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:198 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 65976 1 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23b70fb 0x23b4c5b 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23b70fa github.com/openshift/library-go/pkg/operator/revisioncontroller.StaticPodLatestRevisionClient.UpdateLatestRevisionOperatorStatus+0x29a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/staticpod.go:36 # 0x23b4c5a github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x55a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:115 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 64262 2 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23b6ca5 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23b6ca4 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x444 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 62646 1 @ 0x8f4eca 0x8f4e9f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1895249 0x1895be6 0x1ab1d34 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x8f4ec9 sync.(*Mutex).Lock+0x129 sync/mutex.go:90 # 0x8f4e9e golang.org/x/net/http2.(*clientConnPool).getClientConn+0xfe golang.org/x/net@v0.12.0/http2/client_conn_pool.go:76 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1895248 k8s.io/client-go/kubernetes/typed/core/v1.(*events).PatchWithEventNamespace+0x188 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:99 # 0x1895be5 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Patch+0x25 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:168 # 0x1ab1d33 k8s.io/client-go/tools/record.recordEvent+0x93 k8s.io/client-go@v0.27.4/tools/record/event.go:255 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 61976 3 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x18a837a 0x1efffde 0x1f06ed9 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a8379 k8s.io/client-go/kubernetes/typed/core/v1.(*services).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:79 # 0x1efffdd github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceImproved+0x15d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:139 # 0x1f06ed8 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x518 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:114 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 61950 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff4e0f 0x23ea654 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff4e0e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberPromote+0x6e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:189 # 0x23ea653 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x253 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:283 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 61546 2 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23c49f1 0x23c494c 0x23c03f6 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23c49f0 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1050 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23c494b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).ensureInstallerPod+0xfab github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:923 # 0x23c03f5 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xc95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:476 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 61508 2 @ 0x6019c5 0x60197a 0x601238 0x603c89 0x603c4b 0x1f9604f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x6019c4 sync.(*Mutex).Lock+0xc4 sync/mutex.go:90 # 0x601979 k8s.io/klog/v2.(*loggingT).output+0x79 k8s.io/klog/v2@v2.100.1/klog.go:858 # 0x601237 k8s.io/klog/v2.(*loggingT).printfDepth+0x1f7 k8s.io/klog/v2@v2.100.1/klog.go:737 # 0x603c88 k8s.io/klog/v2.(*loggingT).printf+0x68 k8s.io/klog/v2@v2.100.1/klog.go:718 # 0x603c4a k8s.io/klog/v2.Verbose.Infof+0x2a k8s.io/klog/v2@v2.100.1/klog.go:1418 # 0x1f9604e github.com/openshift/library-go/pkg/operator/status.StatusSyncer.Sync+0xfee github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:213 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 61210 1 @ 0x47e26c 0x90048b 0x91d895 0x57c57f 0x57cd4c 0x659c2d 0x658fcb 0x65825e 0x65bf1b 0x9205e5 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x57c57e bufio.(*Reader).fill+0xfe bufio/bufio.go:106 # 0x57cd4b bufio.(*Reader).ReadByte+0x2b bufio/bufio.go:265 # 0x659c2c compress/flate.(*decompressor).moreBits+0x2c compress/flate/inflate.go:696 # 0x658fca compress/flate.(*decompressor).huffmanBlock+0x50a compress/flate/inflate.go:578 # 0x65825d compress/flate.(*decompressor).Read+0x7d compress/flate/inflate.go:347 # 0x65bf1a compress/gzip.(*Reader).Read+0xba compress/gzip/gunzip.go:252 # 0x9205e4 golang.org/x/net/http2.(*gzipReader).Read+0xe4 golang.org/x/net@v0.12.0/http2/transport.go:3072 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 60988 2 @ 0x481716 0x4816f5 0x1070b5a 0x106f662 0x107e822 0x1ff7fd2 0x1ff3805 0x1ff8bda 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x481715 sync.(*Mutex).Lock+0x35 sync/mutex.go:90 # 0x4816f4 sync.(*RWMutex).Lock+0x14 sync/rwmutex.go:147 # 0x1070b59 k8s.io/client-go/tools/cache.(*DeltaFIFO).HasSynced+0x39 k8s.io/client-go@v0.27.4/tools/cache/delta_fifo.go:301 # 0x106f661 k8s.io/client-go/tools/cache.(*controller).HasSynced+0x21 k8s.io/client-go@v0.27.4/tools/cache/controller.go:169 # 0x107e821 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0xa1 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:519 # 0x1ff7fd1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.endpoints+0x71 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:482 # 0x1ff3804 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient.func1+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:71 # 0x1ff8bd9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x39 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:48 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 59866 1 @ 0x1084268 0x108423a 0x1082ac6 0x18b0e4d 0x1a9a02b 0x23b5622 0x23b479b 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x1084267 sync.(*RWMutex).RLock+0x67 sync/rwmutex.go:71 # 0x1084239 k8s.io/client-go/tools/cache.(*threadSafeMap).Get+0x39 k8s.io/client-go@v0.27.4/tools/cache/thread_safe_store.go:239 # 0x1082ac5 k8s.io/client-go/tools/cache.(*cache).GetByKey+0x25 k8s.io/client-go@v0.27.4/tools/cache/store.go:237 # 0x18b0e4c k8s.io/client-go/listers/core/v1.secretNamespaceLister.Get+0x8c k8s.io/client-go@v0.27.4/listers/core/v1/secret.go:91 # 0x1a9a02a github.com/openshift/library-go/pkg/operator/v1helpers.combinedSecretInterface.Get+0xca github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/core_getters.go:105 # 0x23b5621 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.isLatestRevisionCurrent+0x821 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:172 # 0x23b479a github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:88 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 59688 1 @ 0x9115ba 0x911595 0x9162d2 0x916174 0x911fa5 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23f99a5 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x9115b9 sync.(*Mutex).Lock+0x59 sync/mutex.go:90 # 0x911594 golang.org/x/net/http2.(*clientStream).abortRequestBodyWrite+0x34 golang.org/x/net@v0.12.0/http2/transport.go:434 # 0x9162d1 golang.org/x/net/http2.(*ClientConn).RoundTrip.func2+0x51 golang.org/x/net@v0.12.0/http2/transport.go:1254 # 0x916173 golang.org/x/net/http2.(*ClientConn).RoundTrip+0x533 golang.org/x/net@v0.12.0/http2/transport.go:1295 # 0x911fa4 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x1c4 golang.org/x/net@v0.12.0/http2/transport.go:561 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23f99a4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x1c4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:47 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 59580 1 @ 0x9142b7 0x914297 0x8f503f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x9142b6 sync.(*Mutex).Lock+0x56 sync/mutex.go:90 # 0x914296 golang.org/x/net/http2.(*ClientConn).ReserveNewRequest+0x36 golang.org/x/net@v0.12.0/http2/transport.go:882 # 0x8f503e golang.org/x/net/http2.(*clientConnPool).getClientConn+0x29e golang.org/x/net@v0.12.0/http2/client_conn_pool.go:78 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 58432 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x24061c9 0x240d258 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x24061c8 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller.NewFSyncController+0x168 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller/fsync_controller.go:35 # 0x240d257 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x15b7 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:300 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 58208 2 @ 0x950345 0x95031e 0x950a11 0x9508de 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x950344 sync.(*Mutex).Lock+0x44 sync/mutex.go:90 # 0x95031d google.golang.org/grpc/internal/transport.(*recvBuffer).load+0x1d google.golang.org/grpc@v1.51.0/internal/transport/transport.go:126 # 0x950a10 google.golang.org/grpc/internal/transport.(*recvBufferReader).readAdditional+0x50 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:219 # 0x9508dd google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xdd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:214 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 57754 1 @ 0x987fd4 0x987fb6 0x98358b 0x98357a 0x98303c 0x475761 # 0x987fd3 sync.(*Mutex).Lock+0x53 sync/mutex.go:90 # 0x987fb5 google.golang.org/grpc.(*connectivityStateManager).getState+0x35 google.golang.org/grpc@v1.51.0/clientconn.go:427 # 0x98358a google.golang.org/grpc.(*ClientConn).GetState+0x2a google.golang.org/grpc@v1.51.0/clientconn.go:530 # 0x983579 google.golang.org/grpc.(*ccBalancerWrapper).handleExitIdle+0x19 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:226 # 0x98303b google.golang.org/grpc.(*ccBalancerWrapper).watcher+0x23b google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:127 57490 2 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 55270 4 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 55092 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23d7f26 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d7f25 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Run+0xa5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:440 51848 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x23f91c2 0x23f885a 0x1f8f786 0x1b270e7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f91c1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x5e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:151 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 50376 3 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e7342 0x23e60d4 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e7341 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).canRemoveEtcdBootstrap+0x41 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:181 # 0x23e60d3 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.(*BootstrapTeardownController).sync+0x173 github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:64 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 49794 5 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18a837a 0x1efffde 0x1f06ed9 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a8379 k8s.io/client-go/kubernetes/typed/core/v1.(*services).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/service.go:79 # 0x1efffdd github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceImproved+0x15d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:139 # 0x1f06ed8 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x518 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:114 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 48150 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b23fb7 0x23a93a6 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b23fb6 github.com/openshift/library-go/pkg/operator/events.NewKubeRecorderWithOptions+0x176 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:26 # 0x23a93a5 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x405 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:233 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 48042 2 @ 0x91da45 0x91da19 0x4e4f5a 0x104142d 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x104142c k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Read+0x12c k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:92 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 47040 1 @ 0x1084268 0x108423a 0x1082ac6 0x18af26d 0x23cb6c6 0x23caeb6 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x1084267 sync.(*RWMutex).RLock+0x67 sync/rwmutex.go:71 # 0x1084239 k8s.io/client-go/tools/cache.(*threadSafeMap).Get+0x39 k8s.io/client-go@v0.27.4/tools/cache/thread_safe_store.go:239 # 0x1082ac5 k8s.io/client-go/tools/cache.(*cache).GetByKey+0x25 k8s.io/client-go@v0.27.4/tools/cache/store.go:237 # 0x18af26c k8s.io/client-go/listers/core/v1.podNamespaceLister.Get+0x8c k8s.io/client-go@v0.27.4/listers/core/v1/pod.go:91 # 0x23cb6c5 github.com/openshift/library-go/pkg/operator/staticpod/controller/missingstaticpod.(*missingStaticPodController).getStaticPodCurrentRevisionForNode+0x65 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/missingstaticpod/missing_static_pod_controller.go:187 # 0x23caeb5 github.com/openshift/library-go/pkg/operator/staticpod/controller/missingstaticpod.(*missingStaticPodController).sync+0x3b5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/missingstaticpod/missing_static_pod_controller.go:147 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 46528 2 @ 0x92c872 0x92c848 0x93b038 0x9a32ca 0x9a27da 0x9a41c4 0x9a2236 0x9a153f 0x9a0fee 0x9851fd 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb206a9 0xb96523 0xb8ee88 0xb8e7f0 0x1ffa1ed 0x1ff9ebf 0x475761 # 0x92c871 sync.(*Mutex).Lock+0x51 sync/mutex.go:90 # 0x92c847 google.golang.org/grpc/internal/transport.(*controlBuffer).executeAndPut+0x27 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:331 # 0x93b037 google.golang.org/grpc/internal/transport.(*http2Client).NewStream+0x437 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:803 # 0x9a32c9 google.golang.org/grpc.(*csAttempt).newStream+0x49 google.golang.org/grpc@v1.51.0/stream.go:458 # 0x9a27d9 google.golang.org/grpc.newClientStreamWithParams.func2+0x39 google.golang.org/grpc@v1.51.0/stream.go:327 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a2235 google.golang.org/grpc.newClientStreamWithParams+0xc75 google.golang.org/grpc@v1.51.0/stream.go:336 # 0x9a153e google.golang.org/grpc.newClientStream.func2+0x9e google.golang.org/grpc@v1.51.0/stream.go:193 # 0x9a0fed google.golang.org/grpc.newClientStream+0x4cd google.golang.org/grpc@v1.51.0/stream.go:228 # 0x9851fc google.golang.org/grpc.invoke+0x7c google.golang.org/grpc@v1.51.0/call.go:66 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb206a8 go.etcd.io/etcd/api/v3/etcdserverpb.(*kVClient).Range+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:6460 # 0xb96522 go.etcd.io/etcd/client/v3.(*retryKVClient).Range+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:105 # 0xb8ee87 go.etcd.io/etcd/client/v3.(*kv).Do+0x1e7 go.etcd.io/etcd/client/v3@v3.5.7/kv.go:149 # 0xb8e7ef go.etcd.io/etcd/client/v3.(*kv).Get+0xcf go.etcd.io/etcd/client/v3@v3.5.7/kv.go:119 # 0x1ffa1ec github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x28c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:124 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 45128 3 @ 0x92c872 0x92c848 0x93ec98 0x93ec5c 0x94185f 0x475761 # 0x92c871 sync.(*Mutex).Lock+0x51 sync/mutex.go:90 # 0x92c847 google.golang.org/grpc/internal/transport.(*controlBuffer).executeAndPut+0x27 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:331 # 0x93ec97 google.golang.org/grpc/internal/transport.(*controlBuffer).put+0x77 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:325 # 0x93ec5b google.golang.org/grpc/internal/transport.(*http2Client).handleWindowUpdate+0x3b google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:1330 # 0x94185e google.golang.org/grpc/internal/transport.(*http2Client).reader+0x69e google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:1606 44706 3 @ 0x91da45 0x91da19 0x8770b3 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x193e2b4 0x20b6ccb 0x20b962f 0x20b8f25 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x20b8c8d 0xc05afe 0xc059b6 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x8770b2 net/http.(*cancelTimerBody).Read+0x32 net/http/client.go:955 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193e2b3 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:137 # 0x20b6cca k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Update+0x28a k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:75 # 0x20b962e k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x42e k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:363 # 0x20b8f24 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1+0x24 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:270 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x20b8c8c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1+0x10c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:269 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 44536 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1f1455e 0x2407776 0x240c6e5 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1f1455d github.com/openshift/library-go/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x1bd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:73 # 0x2407775 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x215 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller/resourcesynccontroller.go:19 # 0x240c6e4 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xa44 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:178 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 44030 2 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18aa23a 0x1f00fa5 0x1f070db 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18aa239 k8s.io/client-go/kubernetes/typed/core/v1.(*serviceAccounts).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/serviceaccount.go:81 # 0x1f00fa4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceAccountImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:229 # 0x1f070da github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x71a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:126 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 42232 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb206a9 0xb96523 0xb8ee88 0xb8e7f0 0x1ffa1b0 0x1ff9ebf 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb206a8 go.etcd.io/etcd/api/v3/etcdserverpb.(*kVClient).Range+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:6460 # 0xb96522 go.etcd.io/etcd/client/v3.(*retryKVClient).Range+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:105 # 0xb8ee87 go.etcd.io/etcd/client/v3.(*kv).Do+0x1e7 go.etcd.io/etcd/client/v3@v3.5.7/kv.go:149 # 0xb8e7ef go.etcd.io/etcd/client/v3.(*kv).Get+0xcf go.etcd.io/etcd/client/v3@v3.5.7/kv.go:119 # 0x1ffa1af github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0x24f github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:121 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 42074 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23e1fae 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 41924 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x1b21d12 0x1b2262a 0x23a921a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1b21d11 github.com/openshift/library-go/pkg/operator/events.GetControllerReferenceForCurrentPod+0x111 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder.go:76 # 0x1b22629 github.com/openshift/library-go/pkg/operator/events.GetControllerReferenceForCurrentPod+0xa29 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder.go:61 # 0x23a9219 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x279 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:228 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 41826 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22c49 0xb973d0 0xb8d86e 0x1ff4f0c 0x23ea654 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22c48 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberPromote+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7110 # 0xb973cf go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberPromote+0x2f go.etcd.io/etcd/client/v3@v3.5.7/retry.go:183 # 0xb8d86d go.etcd.io/etcd/client/v3.(*cluster).MemberPromote+0x6d go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:136 # 0x1ff4f0b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberPromote+0x16b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:211 # 0x23ea653 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x253 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:283 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 41508 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x1f94ff7 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x1f94ff6 github.com/openshift/library-go/pkg/operator/status.(*StatusSyncer).Run+0x216 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:102 41444 1 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1894cec 0x1895b22 0x1ab1e0a 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1894ceb k8s.io/client-go/kubernetes/typed/core/v1.(*events).CreateWithEventNamespace+0x10b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:59 # 0x1895b21 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Create+0x21 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:160 # 0x1ab1e09 k8s.io/client-go/tools/record.recordEvent+0x169 k8s.io/client-go@v0.27.4/tools/record/event.go:261 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 40108 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a6985 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6984 google.golang.org/grpc.(*csAttempt).recvMsg+0x724 google.golang.org/grpc@v1.51.0/stream.go:1079 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 39536 2 @ 0x916785 0x91676d 0x9165be 0x475761 # 0x916784 sync.(*Mutex).Lock+0x184 sync/mutex.go:90 # 0x91676c golang.org/x/net/http2.(*clientStream).writeRequest+0x16c golang.org/x/net@v0.12.0/http2/transport.go:1356 # 0x9165bd golang.org/x/net/http2.(*clientStream).doRequest+0x1d golang.org/x/net@v0.12.0/http2/transport.go:1323 39252 2 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23bc1f4 0x23bc195 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23bc1f3 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x1d53 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23bc194 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x1cf4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:337 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 38684 2 @ 0x451705 0x91e052 0x87722a 0x1064a44 0x105f972 0x105f685 0x105fce9 0x193da1a 0x20b6511 0x20b937d 0x20b8f25 0xc086db 0xc088d7 0xc077d8 0xc07629 0xc0759f 0x20b8c8d 0xc05afe 0xc059b6 0xc058a9 0x20b8ae5 0x20b8a5d 0x20b8239 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x877229 net/http.(*cancelTimerBody).Close+0x29 net/http/client.go:972 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193da19 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:78 # 0x20b6510 k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Get+0x90 k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:42 # 0x20b937c k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x17c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:324 # 0x20b8f24 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1.1+0x24 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:270 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc077d7 k8s.io/apimachinery/pkg/util/wait.poll+0x37 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:245 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x20b8c8c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew.func1+0x10c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:269 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b8ae4 k8s.io/apimachinery/pkg/util/wait.Until+0x124 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 # 0x20b8a5c k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew+0x9c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:266 # 0x20b8238 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x118 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:209 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 37950 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 37502 2 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23ea48b 0x23e8f9c 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23ea48a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).ensureEtcdLearnerPromotion+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:254 # 0x23e8f9b github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x2db github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:127 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 36760 1 @ 0x40c378 0x98b605 0x98b5e6 0xb88ad5 0x1ffc191 0x1ff942e 0x1ff5c91 0x23f3504 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x98b604 google.golang.org/grpc.(*ccBalancerWrapper).close+0x164 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:294 # 0x98b5e5 google.golang.org/grpc.(*ClientConn).Close+0x145 google.golang.org/grpc@v1.51.0/clientconn.go:1036 # 0xb88ad4 go.etcd.io/etcd/client/v3.(*Client).Close+0x74 go.etcd.io/etcd/client/v3@v3.5.7/client.go:151 # 0x1ffc190 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func2+0x70 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:158 # 0x1ff942d github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Return+0x8d github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:121 # 0x1ff5c90 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x190 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:277 # 0x23f3503 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x1c3 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:129 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 36428 1 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c0e89 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c0e88 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0x1728 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:501 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 36332 2 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x2408be5 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x2408be4 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x204 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:73 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 35818 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x1f1789c 0x23f27a6 0x23f1c8c 0x240c779 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x1f1789b github.com/openshift/library-go/pkg/operator/configobserver.NewNestedConfigObserver+0x37b github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:114 # 0x23f27a5 github.com/openshift/library-go/pkg/operator/configobserver.NewConfigObserver+0x11a5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:67 # 0x23f1c8b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller.NewConfigObserver+0x68b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller/observe_config_controller.go:59 # 0x240c778 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xad8 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:188 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 35384 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23b466b 0x23da633 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23b466a github.com/openshift/library-go/pkg/operator/revisioncontroller.NewRevisionController+0x3ca github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:82 # 0x23da632 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x552 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:207 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 34268 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 32864 5 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240b5d0 0x240b5d1 0x240a5e5 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240b5cf github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240b5d0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageRecoveryPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:201 # 0x240a5e4 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x284 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:133 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 32388 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23f8750 0x240d60a 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f874f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.NewEtcdEndpointsController+0x5cf github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:67 # 0x240d609 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1969 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:338 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 31912 3 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5b5c 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5b5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0x5b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:263 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 30928 1 @ 0x91ab8d 0x91ab66 0x917cff 0x9165ce 0x475761 # 0x91ab8c sync.(*Mutex).Lock+0x6c sync/mutex.go:90 # 0x91ab65 golang.org/x/net/http2.(*ClientConn).forgetStreamID+0x45 golang.org/x/net@v0.12.0/http2/transport.go:2109 # 0x917cfe golang.org/x/net/http2.(*clientStream).cleanupWriteRequest+0x39e golang.org/x/net@v0.12.0/http2/transport.go:1583 # 0x9165cd golang.org/x/net/http2.(*clientStream).doRequest+0x2d golang.org/x/net@v0.12.0/http2/transport.go:1324 30488 1 @ 0x6019c5 0x60197a 0x601238 0x1b2568d 0x1b2563f 0x1b255e9 0x1b2587d 0x1f95012 0x475761 # 0x6019c4 sync.(*Mutex).Lock+0xc4 sync/mutex.go:90 # 0x601979 k8s.io/klog/v2.(*loggingT).output+0x79 k8s.io/klog/v2@v2.100.1/klog.go:858 # 0x601237 k8s.io/klog/v2.(*loggingT).printfDepth+0x1f7 k8s.io/klog/v2@v2.100.1/klog.go:737 # 0x1b2568c k8s.io/klog/v2.(*loggingT).printf+0x1cc k8s.io/klog/v2@v2.100.1/klog.go:718 # 0x1b2563e k8s.io/klog/v2.Infof+0x17e k8s.io/klog/v2@v2.100.1/klog.go:1494 # 0x1b255e8 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x128 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:73 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 # 0x1f95011 github.com/openshift/library-go/pkg/operator/status.(*StatusSyncer).Run+0x231 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:102 29740 2 @ 0x40c378 0x98b605 0x98b5e6 0xb88ad5 0x1ffc191 0x1ff942e 0x1ff6994 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x98b604 google.golang.org/grpc.(*ccBalancerWrapper).close+0x164 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:294 # 0x98b5e5 google.golang.org/grpc.(*ClientConn).Close+0x145 google.golang.org/grpc@v1.51.0/clientconn.go:1036 # 0xb88ad4 go.etcd.io/etcd/client/v3.(*Client).Close+0x74 go.etcd.io/etcd/client/v3@v3.5.7/client.go:151 # 0x1ffc190 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func2+0x70 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:158 # 0x1ff942d github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Return+0x8d github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:121 # 0x1ff6993 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x593 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:354 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 28676 2 @ 0x4a46de 0x4a46be 0x4a5849 0x4a54d5 0x54cf09 0x54daeb 0x5514c5 0x475761 # 0x4a46dd sync.(*Mutex).Lock+0x5d sync/mutex.go:90 # 0x4a46bd context.removeChild+0x3d context/context.go:373 # 0x4a5848 context.(*timerCtx).cancel+0x68 context/context.go:546 # 0x4a54d4 context.WithDeadline.func3+0x34 context/context.go:519 # 0x54cf08 net.(*Resolver).exchange+0x8e8 net/dnsclient_unix.go:201 # 0x54daea net.(*Resolver).tryOneName+0x46a net/dnsclient_unix.go:277 # 0x5514c4 net.(*Resolver).goLookupIPCNAMEOrder.func3.1+0x84 net/dnsclient_unix.go:653 28600 32 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2312863 0x480462 0x2311371 0x2311355 0x233bf0a 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2312862 k8s.io/apiserver/pkg/util/flowcontrol/metrics.Register.func1+0x42 k8s.io/apiserver@v0.27.4/pkg/util/flowcontrol/metrics/metrics.go:59 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2311370 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2311354 k8s.io/apiserver/pkg/util/flowcontrol/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/util/flowcontrol/metrics/metrics.go:57 # 0x233bf09 k8s.io/apiserver/pkg/server/routes.register+0x29 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:52 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 28530 3 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x1ff5dea 0x23e3298 0x23eb49f 0x23e940b 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x1ff5de9 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).VotingMemberList+0x29 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:281 # 0x23e3297 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.VotingMemberIPListSet+0x57 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/common.go:183 # 0x23eb49e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).allNodesMapToVotingMembers+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:382 # 0x23e940a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x8a github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:142 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 27804 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x20ba582 0x23aa42b 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x20ba581 github.com/openshift/library-go/pkg/config/leaderelection.ToLeaderElectionWithLease+0x241 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/leaderelection/leaderelection.go:60 # 0x23aa42a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x148a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:332 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 26908 1 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 26812 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x2409928 0x240cd9f 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x2409927 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.NewTargetConfigController+0x1e7 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:68 # 0x240cd9e github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x10fe github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:228 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 26772 2 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be75a5 0x1f0abeb 0x1f127bf 0x1f07deb 0x23d6965 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be75a4 k8s.io/client-go/dynamic.(*dynamicResourceClient).Get+0x544 k8s.io/client-go@v0.27.4/dynamic/simple.go:280 # 0x1f0abea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyServiceMonitor+0x20a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/monitoring.go:60 # 0x1f127be github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyKnownUnstructured+0x1be github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/unstructured.go:18 # 0x1f07dea github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyDirectly+0x142a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/generic.go:212 # 0x23d6964 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Sync+0x544 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:328 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 26372 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1ff3409 0x240c685 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1ff3408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient+0x1e8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:67 # 0x240c684 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x9e4 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:173 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 26196 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23d83df 0x240df5b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d83de github.com/openshift/library-go/pkg/operator/unsupportedconfigoverridescontroller.NewUnsupportedConfigOverridesController+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/unsupportedconfigoverridescontroller/unsupportedconfigoverrides_controller.go:35 # 0x240df5a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x22ba github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:438 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 26006 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23f8731 0x240d60a 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f8730 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.NewEtcdEndpointsController+0x5b0 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:67 # 0x240d609 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1969 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:338 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 25712 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23bdcd8 0x23b30e8 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23bdcd7 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).Run+0xd7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:198 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 25440 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x1b276e5 0x1b2710b 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x1b276e4 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reportDegraded+0x464 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:236 # 0x1b2710a github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x6a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:202 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 25344 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23d7f26 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d7f25 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Run+0xa5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:440 25122 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x24084e7 0x240dc5b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x24084e6 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.NewScriptControllerController+0x226 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:52 # 0x240dc5a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1fba github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:406 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 24676 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c6b8e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c6b8d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x4cd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1084 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 24350 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23ddefe 0x23de8ef 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23ddefd github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).sync+0x25d github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:123 # 0x23de8ee github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x10e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:204 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 24072 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1890274 0x1f02630 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1890273 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Update+0x173 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:137 # 0x1f0262f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0x102f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:333 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 24050 2 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 24038 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x1f94ff7 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x1f94ff6 github.com/openshift/library-go/pkg/operator/status.(*StatusSyncer).Run+0x216 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:102 23974 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x24061c9 0x240d258 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x24061c8 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller.NewFSyncController+0x168 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller/fsync_controller.go:35 # 0x240d257 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x15b7 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:300 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 23126 1 @ 0x9142b7 0x914297 0x8f503f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x188fc30 0x1a449a5 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x9142b6 sync.(*Mutex).Lock+0x56 sync/mutex.go:90 # 0x914296 golang.org/x/net/http2.(*ClientConn).ReserveNewRequest+0x36 golang.org/x/net@v0.12.0/http2/transport.go:882 # 0x8f503e golang.org/x/net/http2.(*clientConnPool).getClientConn+0x29e golang.org/x/net@v0.12.0/http2/client_conn_pool.go:78 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188fc2f k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:95 # 0x1a449a4 k8s.io/client-go/informers/core/v1.NewFilteredConfigMapInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 23040 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23d22d3 0x23dad85 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d22d2 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.NewStaticPodStateController+0x412 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:61 # 0x23dad84 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0xca4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:258 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 22994 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23d7f26 0x23b30e8 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d7f25 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Run+0xa5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:440 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 22954 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23b466b 0x23da633 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23b466a github.com/openshift/library-go/pkg/operator/revisioncontroller.NewRevisionController+0x3ca github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:82 # 0x23da632 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x552 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:207 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 22564 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23e5eaf 0x240dbec 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23e5eae github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.NewBootstrapTeardownController+0x4ee github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:52 # 0x240dbeb github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1f4b github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:397 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 22404 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23f9762 0x240db52 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f9761 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.NewEtcdMembersController+0x1e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:41 # 0x240db51 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1eb1 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:390 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 22138 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x20ba4af 0x23aa42b 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x20ba4ae github.com/openshift/library-go/pkg/config/leaderelection.ToLeaderElectionWithLease+0x16e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/config/leaderelection/leaderelection.go:59 # 0x23aa42a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x148a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:332 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 21986 1 @ 0x47fefb 0x47e27e 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x47e27d sync.(*Cond).Wait+0x9d sync/cond.go:71 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 21876 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23cc225 0x23db1d1 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23cc224 github.com/openshift/library-go/pkg/operator/staticpod/controller/node.NewNodeController+0x2e4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/node/node_controller.go:38 # 0x23db1d0 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x10f0 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:313 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 21870 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x18a733c 0x1f030b3 0x23f7534 0x23f74ef 0x23f6a49 0x23f6417 0x23f60f9 0x23f5db6 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a733b k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).Create+0x15b k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:123 # 0x1f030b2 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecretImproved+0x6f2 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:378 # 0x23f7533 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplySecret+0x813 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:87 # 0x23f74ee github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).ensureCertSecret+0x7ce github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:291 # 0x23f6a48 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).ensureCertsForNode+0x2a8 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:227 # 0x23f6416 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).ensureCerts+0x1f6 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:195 # 0x23f60f8 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).syncAllMasters+0x38 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:152 # 0x23f5db5 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0xf5 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:127 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 21760 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23d3cd8 0x23db325 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23d3cd7 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.NewStaticResourceController+0x97 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:112 # 0x23db324 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x1244 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:320 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 21664 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23f5c3a 0x240d578 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f5c39 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.NewEtcdCertSignerController+0x6f9 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:114 # 0x240d577 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x18d7 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:329 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 21560 19 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x22e2f86 0x480462 0x22de171 0x22de155 0x233bef7 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x22e2f85 k8s.io/apiserver/pkg/endpoints/metrics.Register.func1+0x45 k8s.io/apiserver@v0.27.4/pkg/endpoints/metrics/metrics.go:390 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x22de170 sync.(*Once).Do+0x30 sync/once.go:65 # 0x22de154 k8s.io/apiserver/pkg/endpoints/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/endpoints/metrics/metrics.go:388 # 0x233bef6 k8s.io/apiserver/pkg/server/routes.register+0x16 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:49 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 21334 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23dd2ae 0x240c9ee 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23dd2ad github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.NewEnvVarController+0x44d github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:79 # 0x240c9ed github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xd4d github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:212 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 20510 4 @ 0x92c872 0x92c848 0x93cb45 0x93cb1e 0x9a5fbc 0x9a5233 0x9a41c4 0x9a4ea5 0x985228 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x92c871 sync.(*Mutex).Lock+0x51 sync/mutex.go:90 # 0x92c847 google.golang.org/grpc/internal/transport.(*controlBuffer).executeAndPut+0x27 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:331 # 0x93cb44 google.golang.org/grpc/internal/transport.(*controlBuffer).put+0x1c4 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:325 # 0x93cb1d google.golang.org/grpc/internal/transport.(*http2Client).Write+0x19d google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:1021 # 0x9a5fbb google.golang.org/grpc.(*csAttempt).sendMsg+0x2db google.golang.org/grpc@v1.51.0/stream.go:1001 # 0x9a5232 google.golang.org/grpc.(*clientStream).SendMsg.func2+0x52 google.golang.org/grpc@v1.51.0/stream.go:856 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a4ea4 google.golang.org/grpc.(*clientStream).SendMsg+0x324 google.golang.org/grpc@v1.51.0/stream.go:858 # 0x985227 google.golang.org/grpc.invoke+0xa7 google.golang.org/grpc@v1.51.0/call.go:70 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 20424 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23b0b0a 0x1a9b5b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9b366 0x1a9b29d 0x23c0e89 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23b0b09 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.UpdateStaticPodOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:119 # 0x1a9b5b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:223 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9b365 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9b29c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStaticPodStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:204 # 0x23c0e88 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0x1728 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:501 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 20402 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23ec4d3 0x240da07 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23ec4d2 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.NewClusterMemberRemovalController+0x852 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:98 # 0x240da06 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1d66 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:364 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 19678 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x1ffd7da 0x240df17 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x1ffd7d9 github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller.NewUpgradeBackupController+0x6d9 github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller/upgradebackupcontroller.go:100 # 0x240df16 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x2276 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:424 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 19624 1 @ 0x47e26c 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x1b27465 0x1b2710b 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47e26b sync.(*Cond).Wait+0x8b sync/cond.go:70 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x1b27464 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reportDegraded+0x1e4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:225 # 0x1b2710a github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x6a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:202 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 19460 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23c25f4 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23c25f3 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0x2d3 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:687 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 19316 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23ec4f2 0x240da07 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23ec4f1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.NewClusterMemberRemovalController+0x871 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:98 # 0x240da06 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1d66 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:364 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 19184 1 @ 0x90040e 0x9003f3 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23beb5c 0x23bf85f 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x90040d sync.(*Mutex).Lock+0x6d sync/mutex.go:90 # 0x9003f2 golang.org/x/net/http2.(*pipe).Read+0x52 golang.org/x/net@v0.12.0/http2/pipe.go:56 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23beb5b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.nodeToStartRevisionWith+0x2db github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:319 # 0x23bf85e github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xfe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:434 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 18978 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23d22d3 0x23dad85 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d22d2 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.NewStaticPodStateController+0x412 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:61 # 0x23dad84 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0xca4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:258 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 18956 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23bdcd8 0x23b30e8 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23bdcd7 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).Run+0xd7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:198 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 18548 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23fa3e5 0x23f985a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23fa3e4 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).reportEtcdMembers+0x844 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:132 # 0x23f9859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.(*EtcdMembersController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:45 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 18518 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23ba16e 0x23db66b 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23ba16d github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.NewGuardController+0x68d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:95 # 0x23db66a github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x158a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:336 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 18416 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23e5eaf 0x240dbec 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23e5eae github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.NewBootstrapTeardownController+0x4ee github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:52 # 0x240dbeb github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1f4b github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:397 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 18258 1 @ 0x1084268 0x108423a 0x1082ac6 0x1f7a48f 0x1ffd8ee 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x1084267 sync.(*RWMutex).RLock+0x67 sync/rwmutex.go:71 # 0x1084239 k8s.io/client-go/tools/cache.(*threadSafeMap).Get+0x39 k8s.io/client-go@v0.27.4/tools/cache/thread_safe_store.go:239 # 0x1082ac5 k8s.io/client-go/tools/cache.(*cache).GetByKey+0x25 k8s.io/client-go@v0.27.4/tools/cache/store.go:237 # 0x1f7a48e github.com/openshift/client-go/config/listers/config/v1.(*clusterOperatorLister).Get+0x2e github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/listers/config/v1/clusteroperator.go:44 # 0x1ffd8ed github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller.(*UpgradeBackupController).sync+0x6d github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller/upgradebackupcontroller.go:104 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 18034 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23dd2ae 0x240c9ee 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23dd2ad github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.NewEnvVarController+0x44d github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:79 # 0x240c9ed github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xd4d github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:212 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 17928 4 @ 0x40b61d 0x107f652 0x107f689 0x107ed87 0x1b2a9ac 0x23d7f26 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x107f651 k8s.io/client-go/tools/cache.(*processorListener).add+0x671 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x107f688 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandlerWithResyncPeriod+0x6a8 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:628 # 0x107ed86 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandler+0x26 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:556 # 0x1b2a9ab github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x9cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:294 # 0x23d7f25 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Run+0xa5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:440 17716 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23f2cf1 0x240dd2b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f2cf0 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.NewDefragController+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:74 # 0x240dd2a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x208a github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:415 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 17688 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x23dde18 0x23de8ef 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x23dde17 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).sync+0x177 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:111 # 0x23de8ee github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x10e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:204 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 17676 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1f4c419 0x1f7d646 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1f4c418 github.com/openshift/client-go/config/clientset/versioned/typed/config/v1.(*aPIServers).List+0x158 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/clientset/versioned/typed/config/v1/apiserver.go:77 # 0x1f7d645 github.com/openshift/client-go/config/informers/externalversions/config/v1.NewFilteredAPIServerInformer.func1+0x125 github.com/openshift/client-go@v0.0.0-20230718103934-90d8d813cf54/config/informers/externalversions/config/v1/apiserver.go:48 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 17606 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23e5ece 0x240dbec 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23e5ecd github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.NewBootstrapTeardownController+0x50d github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:52 # 0x240dbeb github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1f4b github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:397 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 17550 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x240993f 0x240cd9f 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x240993e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.NewTargetConfigController+0x1fe github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:68 # 0x240cd9e github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x10fe github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:228 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 17332 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23e8b4f 0x240d8b2 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23e8b4e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.NewClusterMemberController+0x98e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:95 # 0x240d8b1 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1c11 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:350 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 17116 1 @ 0x451705 0x9508a5 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff726d 0x23e1fae 0x23e380c 0x240a11f 0x1f8f786 0x1b270e7 0x1b2792c # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x9508a4 google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xa4 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:195 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff726c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x6c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:399 # 0x23e1fad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x2ad github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:150 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x240a11e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:89 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 16880 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23d7f26 0x23b30e8 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d7f25 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.(*StaticResourceController).Run+0xa5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:440 # 0x23b30e7 github.com/openshift/library-go/pkg/controller/manager.controllerManager.Start.func1+0x107 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/manager/controller_manager.go:53 16878 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23b3575 0x23db3ff 0x23db3b2 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23b3574 github.com/openshift/library-go/pkg/operator/loglevel.NewClusterOperatorLoggingControllerWithLogLevel+0x254 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/loglevel/logging_controller.go:40 # 0x23db3fe github.com/openshift/library-go/pkg/operator/loglevel.NewClusterOperatorLoggingController+0x131e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/loglevel/logging_controller.go:29 # 0x23db3b1 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x12d1 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:333 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16868 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23f5c1b 0x240d578 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f5c1a github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.NewEtcdCertSignerController+0x6da github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:114 # 0x240d577 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x18d7 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:329 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16864 1 @ 0x101aaa5 0x101aa87 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x101aaa4 sync.(*Mutex).Lock+0x84 sync/mutex.go:90 # 0x101aa86 golang.org/x/time/rate.(*Limiter).WaitN+0x66 golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:224 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 16802 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1ff3409 0x240c685 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1ff3408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewEtcdClient+0x1e8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:67 # 0x240c684 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x9e4 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:173 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16754 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1f94b42 0x240d3aa 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f94b41 github.com/openshift/library-go/pkg/operator/status.NewClusterOperatorStatusController+0x281 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:82 # 0x240d3a9 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1709 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:305 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16730 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23c7da5 0x23dab45 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23c7da4 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.NewInstallerStateController+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:53 # 0x23dab44 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0xa64 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:244 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16686 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23fb365 0x240daf4 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23fb364 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.NewMachineDeletionHooksController+0x664 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:82 # 0x240daf3 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1e53 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:378 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16630 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23cdaf0 0x23daf37 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23cdaef github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.NewPruneController+0x42f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:77 # 0x23daf36 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0xe56 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:274 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16600 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23caa86 0x23db81b 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23caa85 github.com/openshift/library-go/pkg/operator/staticpod/controller/missingstaticpod.New+0x6c5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/missingstaticpod/missing_static_pod_controller.go:91 # 0x23db81a github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x173a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:357 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16426 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x24084fe 0x240dc5b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x24084fd github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.NewScriptControllerController+0x23d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:52 # 0x240dc5a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1fba github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:406 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16226 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23ec4f2 0x240da07 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23ec4f1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.NewClusterMemberRemovalController+0x871 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:98 # 0x240da06 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1d66 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:364 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16106 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1f94b42 0x240d3aa 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f94b41 github.com/openshift/library-go/pkg/operator/status.NewClusterOperatorStatusController+0x281 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:82 # 0x240d3a9 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1709 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:305 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 16010 1 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff7317 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff7316 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x116 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:408 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 15988 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23fb344 0x240daf4 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23fb343 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.NewMachineDeletionHooksController+0x643 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:82 # 0x240daf3 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1e53 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:378 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15862 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23c7da5 0x23dab45 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23c7da4 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.NewInstallerStateController+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:53 # 0x23dab44 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0xa64 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:244 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15860 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23caa86 0x23db81b 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23caa85 github.com/openshift/library-go/pkg/operator/staticpod/controller/missingstaticpod.New+0x6c5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/missingstaticpod/missing_static_pod_controller.go:91 # 0x23db81a github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x173a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:357 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15752 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23ec4d3 0x240da07 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23ec4d2 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller.NewClusterMemberRemovalController+0x852 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermemberremovalcontroller/clustermemberremovalcontroller.go:98 # 0x240da06 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1d66 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:364 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15748 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x240993f 0x240cd9f 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x240993e github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.NewTargetConfigController+0x1fe github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:68 # 0x240cd9e github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x10fe github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:228 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15708 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23e8b6e 0x240d8b2 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23e8b6d github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.NewClusterMemberController+0x9ad github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:95 # 0x240d8b1 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1c11 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:350 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15636 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23d83df 0x23db399 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d83de github.com/openshift/library-go/pkg/operator/unsupportedconfigoverridescontroller.NewUnsupportedConfigOverridesController+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/unsupportedconfigoverridescontroller/unsupportedconfigoverrides_controller.go:35 # 0x23db398 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x12b8 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:332 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15608 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x2409928 0x240cd9f 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x2409927 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.NewTargetConfigController+0x1e7 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:68 # 0x240cd9e github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x10fe github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:228 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15452 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23f2cf1 0x240dd2b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f2cf0 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.NewDefragController+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:74 # 0x240dd2a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x208a github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:415 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15440 3 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff6542 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff6541 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x141 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:337 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 15438 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23b3575 0x23db3ff 0x23db3b2 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23b3574 github.com/openshift/library-go/pkg/operator/loglevel.NewClusterOperatorLoggingControllerWithLogLevel+0x254 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/loglevel/logging_controller.go:40 # 0x23db3fe github.com/openshift/library-go/pkg/operator/loglevel.NewClusterOperatorLoggingController+0x131e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/loglevel/logging_controller.go:29 # 0x23db3b1 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x12d1 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:333 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15412 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1ffd7bb 0x240df17 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1ffd7ba github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller.NewUpgradeBackupController+0x6ba github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller/upgradebackupcontroller.go:100 # 0x240df16 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x2276 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:424 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15378 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23fb365 0x240daf4 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23fb364 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.NewMachineDeletionHooksController+0x664 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:82 # 0x240daf3 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1e53 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:378 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15330 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23f977e 0x240db52 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f977d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.NewEtcdMembersController+0x1fd github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:41 # 0x240db51 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1eb1 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:390 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15168 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23fb344 0x240daf4 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23fb343 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.NewMachineDeletionHooksController+0x643 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:82 # 0x240daf3 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1e53 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:378 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15138 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23d83df 0x23db399 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d83de github.com/openshift/library-go/pkg/operator/unsupportedconfigoverridescontroller.NewUnsupportedConfigOverridesController+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/unsupportedconfigoverridescontroller/unsupportedconfigoverrides_controller.go:35 # 0x23db398 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x12b8 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:332 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15108 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23f8731 0x240d60a 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f8730 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.NewEtcdEndpointsController+0x5b0 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:67 # 0x240d609 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1969 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:338 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 15050 1 @ 0x91db99 0x91db80 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23c25f4 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91db98 sync.(*Mutex).Lock+0x378 sync/mutex.go:90 # 0x91db7f golang.org/x/net/http2.transportResponseBody.Read+0x35f golang.org/x/net@v0.12.0/http2/transport.go:2566 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23c25f3 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0x2d3 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:687 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 14964 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23f977e 0x240db52 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f977d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.NewEtcdMembersController+0x1fd github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:41 # 0x240db51 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1eb1 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:390 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 14956 1 @ 0x451705 0x91e052 0x87722a 0x1064a44 0x105f972 0x105f685 0x105fce9 0x193da1a 0x20b6511 0x20b937d 0x20b87e5 0xc05afe 0xc059b6 0xc058a9 0x20b86fa 0x20b817c 0x20b83d4 0x23aa528 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x877229 net/http.(*cancelTimerBody).Close+0x29 net/http/client.go:972 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x193da19 k8s.io/client-go/kubernetes/typed/coordination/v1.(*leases).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/coordination/v1/lease.go:78 # 0x20b6510 k8s.io/client-go/tools/leaderelection/resourcelock.(*LeaseLock).Get+0x90 k8s.io/client-go@v0.27.4/tools/leaderelection/resourcelock/leaselock.go:42 # 0x20b937c k8s.io/client-go/tools/leaderelection.(*LeaderElector).tryAcquireOrRenew+0x17c k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:324 # 0x20b87e4 k8s.io/client-go/tools/leaderelection.(*LeaderElector).acquire.func1+0x64 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:247 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0x20b86f9 k8s.io/client-go/tools/leaderelection.(*LeaderElector).acquire+0x219 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:246 # 0x20b817b k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run+0x5b k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:203 # 0x20b83d3 k8s.io/client-go/tools/leaderelection.RunOrDie+0x93 k8s.io/client-go@v0.27.4/tools/leaderelection/leaderelection.go:223 # 0x23aa527 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x1587 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:342 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 14856 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1b2816a 0x1b2a065 0x23f2d10 0x240dd2b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f2d0f github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.NewDefragController+0x3cf github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:74 # 0x240dd2a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x208a github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:415 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 14468 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23f5c1b 0x240d578 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f5c1a github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.NewEtcdCertSignerController+0x6da github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:114 # 0x240d577 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x18d7 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:329 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 14422 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23bd5d0 0x23da8e8 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23bd5cf github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.NewInstallerController+0x8f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:179 # 0x23da8e7 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x807 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:222 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 14390 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23cc225 0x23db1d1 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23cc224 github.com/openshift/library-go/pkg/operator/staticpod/controller/node.NewNodeController+0x2e4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/node/node_controller.go:38 # 0x23db1d0 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x10f0 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:313 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 14378 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23f5c3a 0x240d578 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f5c39 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.NewEtcdCertSignerController+0x6f9 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:114 # 0x240d577 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x18d7 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:329 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 14272 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23d3cd8 0x23db325 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23d3cd7 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.NewStaticResourceController+0x97 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:112 # 0x23db324 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x1244 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:320 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 14220 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x1ffd7da 0x240df17 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x1ffd7d9 github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller.NewUpgradeBackupController+0x6d9 github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller/upgradebackupcontroller.go:100 # 0x240df16 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x2276 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:424 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 13842 4 @ 0x4809ff 0x4809da 0x48094e 0x480685 0x93d44e 0x93d439 0x9417de 0x475761 # 0x4809fe sync.(*Mutex).Lock+0x7e sync/mutex.go:90 # 0x4809d9 sync.(*Pool).pinSlow+0x59 sync/pool.go:216 # 0x48094d sync.(*Pool).pin+0x4d sync/pool.go:209 # 0x480684 sync.(*Pool).Get+0x24 sync/pool.go:131 # 0x93d44d google.golang.org/grpc/internal/transport.(*bufferPool).get+0x2ed google.golang.org/grpc@v1.51.0/internal/transport/transport.go:67 # 0x93d438 google.golang.org/grpc/internal/transport.(*http2Client).handleData+0x2d8 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:1126 # 0x9417dd google.golang.org/grpc/internal/transport.(*http2Client).reader+0x61d google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:1596 13680 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23e8b4f 0x240d8b2 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23e8b4e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.NewClusterMemberController+0x98e github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:95 # 0x240d8b1 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1c11 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:350 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 13672 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23f2d10 0x240dd2b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f2d0f github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.NewDefragController+0x3cf github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:74 # 0x240dd2a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x208a github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:415 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 13576 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1f14b8e 0x2407776 0x240c6e5 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f14b8d github.com/openshift/library-go/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x7ed github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:88 # 0x2407775 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x215 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller/resourcesynccontroller.go:19 # 0x240c6e4 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xa44 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:178 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 13174 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x24084fe 0x240dc5b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x24084fd github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.NewScriptControllerController+0x23d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:52 # 0x240dc5a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1fba github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:406 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 13056 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23f8750 0x240d60a 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23f874f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.NewEtcdEndpointsController+0x5cf github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:67 # 0x240d609 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1969 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:338 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 13008 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23bd5d0 0x23da8e8 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23bd5cf github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.NewInstallerController+0x8f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:179 # 0x23da8e7 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x807 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:222 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 12824 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x24084e7 0x240dc5b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x24084e6 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.NewScriptControllerController+0x226 github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:52 # 0x240dc5a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1fba github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:406 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 12760 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23ba16e 0x23db66b 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23ba16d github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.NewGuardController+0x68d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:95 # 0x23db66a github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0x158a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:336 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 12748 1 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23e94a2 0x23e8d51 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23e94a1 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).getEtcdPeerURLToAdd+0x121 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:152 # 0x23e8d50 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x90 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:115 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 12738 1 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x1be5e9a 0x23ae78a 0x1a9b0b6 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x2408d0e 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1be5e99 k8s.io/client-go/dynamic.(*dynamicResourceClient).UpdateStatus+0x5f9 k8s.io/client-go@v0.27.4/dynamic/simple.go:216 # 0x23ae789 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.UpdateOperatorStatus+0x169 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:148 # 0x1a9b0b5 github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x1d5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:181 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x2408d0d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x32d github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:85 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 12730 1 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23beb5c 0x23bf85f 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23beb5b github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.nodeToStartRevisionWith+0x2db github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:319 # 0x23bf85e github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xfe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:434 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 12564 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23cdaf0 0x23daf37 0x240d1d9 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23cdaef github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.NewPruneController+0x42f github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:77 # 0x23daf36 github.com/openshift/library-go/pkg/operator/staticpod.(*staticPodOperatorControllerBuilder).ToControllers+0xe56 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controllers.go:274 # 0x240d1d8 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1538 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:295 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 12296 2 @ 0x47fefb 0x1025547 0x1025485 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x1025546 k8s.io/client-go/util/workqueue.(*Type).updateUnfinishedWorkLoop.func1+0x46 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:316 # 0x1025484 k8s.io/client-go/util/workqueue.(*Type).updateUnfinishedWorkLoop+0xc4 k8s.io/client-go@v0.27.4/util/workqueue/queue.go:324 12230 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23e8b6e 0x240d8b2 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23e8b6d github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.NewClusterMemberController+0x9ad github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:95 # 0x240d8b1 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1c11 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:350 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 12044 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23d83df 0x240df5b 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23d83de github.com/openshift/library-go/pkg/operator/unsupportedconfigoverridescontroller.NewUnsupportedConfigOverridesController+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/unsupportedconfigoverridescontroller/unsupportedconfigoverrides_controller.go:35 # 0x240df5a github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x22ba github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:438 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 11988 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1ffd7bb 0x240df17 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1ffd7ba github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller.NewUpgradeBackupController+0x6ba github.com/openshift/cluster-etcd-operator/pkg/operator/upgradebackupcontroller/upgradebackupcontroller.go:100 # 0x240df16 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x2276 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:424 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 11836 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x23e5ece 0x240dbec 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x23e5ecd github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown.NewBootstrapTeardownController+0x50d github.com/openshift/cluster-etcd-operator/pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go:52 # 0x240dbeb github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1f4b github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:397 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 11728 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23f9762 0x240db52 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23f9761 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller.NewEtcdMembersController+0x1e1 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdmemberscontroller/etcdmemberscontroller.go:41 # 0x240db51 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0x1eb1 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:390 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 11668 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x23d3cd8 0x240c8df 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23d3cd7 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.NewStaticResourceController+0x97 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:112 # 0x240c8de github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xc3e github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:197 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 11646 1 @ 0x92cccd 0x92ccab 0x92d565 0x936d0a 0x475761 # 0x92cccc sync.(*Mutex).Lock+0x6c sync/mutex.go:90 # 0x92ccaa google.golang.org/grpc/internal/transport.(*controlBuffer).get+0x4a google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:382 # 0x92d564 google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x164 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:548 # 0x936d09 google.golang.org/grpc/internal/transport.newHTTP2Client.func6+0x69 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:442 11480 1 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x1895249 0x1895be6 0x1ab1d34 0x1ab19eb 0x1ab17ef 0x1ab2c02 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x1895248 k8s.io/client-go/kubernetes/typed/core/v1.(*events).PatchWithEventNamespace+0x188 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:99 # 0x1895be5 k8s.io/client-go/kubernetes/typed/core/v1.(*EventSinkImpl).Patch+0x25 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/event_expansion.go:168 # 0x1ab1d33 k8s.io/client-go/tools/record.recordEvent+0x93 k8s.io/client-go@v0.27.4/tools/record/event.go:255 # 0x1ab19ea k8s.io/client-go/tools/record.(*eventBroadcasterImpl).recordToSink+0x16a k8s.io/client-go@v0.27.4/tools/record/event.go:223 # 0x1ab17ee k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink.func1+0x2e k8s.io/client-go@v0.27.4/tools/record/event.go:200 # 0x1ab2c01 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1+0xa1 k8s.io/client-go@v0.27.4/tools/record/event.go:327 11308 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x23d3cd8 0x240c8df 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x23d3cd7 github.com/openshift/library-go/pkg/operator/staticresourcecontroller.NewStaticResourceController+0x97 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticresourcecontroller/static_resource_controller.go:112 # 0x240c8de github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xc3e github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:197 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 10726 1 @ 0x40c378 0x98b605 0x98b5e6 0xb88ad5 0x1ffc191 0x1ff942e 0x1ff73d8 0x23f33aa 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x98b604 google.golang.org/grpc.(*ccBalancerWrapper).close+0x164 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:294 # 0x98b5e5 google.golang.org/grpc.(*ClientConn).Close+0x145 google.golang.org/grpc@v1.51.0/clientconn.go:1036 # 0xb88ad4 go.etcd.io/etcd/client/v3.(*Client).Close+0x74 go.etcd.io/etcd/client/v3@v3.5.7/client.go:151 # 0x1ffc190 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func2+0x70 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:158 # 0x1ff942d github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Return+0x8d github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:121 # 0x1ff73d7 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberHealth+0x1d7 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:412 # 0x23f33a9 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x69 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:121 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 10452 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1f14547 0x2407776 0x240c6e5 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f14546 github.com/openshift/library-go/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x1a6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:73 # 0x2407775 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x215 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller/resourcesynccontroller.go:19 # 0x240c6e4 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xa44 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:178 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 9904 2 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x23bb967 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bb966 github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x14c6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:300 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 9540 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1f1455e 0x2407776 0x240c6e5 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1f1455d github.com/openshift/library-go/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x1bd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:73 # 0x2407775 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x215 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller/resourcesynccontroller.go:19 # 0x240c6e4 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xa44 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:178 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 8978 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1f14b8e 0x2407776 0x240c6e5 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f14b8d github.com/openshift/library-go/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x7ed github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:88 # 0x2407775 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x215 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller/resourcesynccontroller.go:19 # 0x240c6e4 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xa44 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:178 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 7742 1 @ 0x47fefb 0x47e27e 0x90048b 0x91d895 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x240ba10 0x240ba11 0x240a47c 0x240a245 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x47fefa sync.(*Mutex).Lock+0x3a sync/mutex.go:90 # 0x47e27d sync.(*Cond).Wait+0x9d sync/cond.go:71 # 0x90048a golang.org/x/net/http2.(*pipe).Read+0xea golang.org/x/net@v0.12.0/http2/pipe.go:76 # 0x91d894 golang.org/x/net/http2.transportResponseBody.Read+0x74 golang.org/x/net@v0.12.0/http2/transport.go:2534 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x240ba0f github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x3af github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240ba10 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.(*TargetConfigController).manageStandardPod+0x3b0 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:215 # 0x240a47b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.createTargetConfig+0x11b github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:128 # 0x240a244 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller.TargetConfigController.sync+0x184 github.com/openshift/cluster-etcd-operator/pkg/operator/targetconfigcontroller/targetconfigcontroller.go:107 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 7696 1 @ 0xd0a58d 0xd0a569 0xd0c310 0xd099f2 0xd08f7c 0x23b0ca5 0x23b0c85 0x23b02a7 0x23cf59c 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0xd0a58c sync.(*Mutex).Lock+0x40c sync/mutex.go:90 # 0xd0a568 k8s.io/apimachinery/pkg/runtime.fieldInfoFromField+0x3e8 k8s.io/apimachinery@v0.27.4/pkg/runtime/converter.go:397 # 0xd0c30f k8s.io/apimachinery/pkg/runtime.structFromUnstructured+0x34f k8s.io/apimachinery@v0.27.4/pkg/runtime/converter.go:528 # 0xd099f1 k8s.io/apimachinery/pkg/runtime.fromUnstructured+0x411 k8s.io/apimachinery@v0.27.4/pkg/runtime/converter.go:359 # 0xd08f7b k8s.io/apimachinery/pkg/runtime.(*unstructuredConverter).FromUnstructuredWithValidation+0x31b k8s.io/apimachinery@v0.27.4/pkg/runtime/converter.go:247 # 0x23b0ca4 k8s.io/apimachinery/pkg/runtime.(*unstructuredConverter).FromUnstructured+0x84 k8s.io/apimachinery@v0.27.4/pkg/runtime/converter.go:274 # 0x23b0c84 github.com/openshift/library-go/pkg/operator/genericoperatorclient.getStaticPodOperatorSpecFromUnstructured+0x64 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:141 # 0x23b02a6 github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicStaticPodOperatorClient.GetStaticPodOperatorState+0xa6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_staticpod_operator_client.go:51 # 0x23cf59b github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0xdb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:255 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 7646 2 @ 0x91da45 0x91da19 0x5f3dc8 0x5f39bb 0x5f35f8 0x1041745 0x10354c7 0x1035bef 0xd2a53c 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x5f3dc7 encoding/json.(*Decoder).refill+0x187 encoding/json/stream.go:165 # 0x5f39ba encoding/json.(*Decoder).readValue+0xba encoding/json/stream.go:140 # 0x5f35f7 encoding/json.(*Decoder).Decode+0x77 encoding/json/stream.go:63 # 0x1041744 k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read+0x1a4 k8s.io/apimachinery@v0.27.4/pkg/util/framer/framer.go:152 # 0x10354c6 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode+0xa6 k8s.io/apimachinery@v0.27.4/pkg/runtime/serializer/streaming/streaming.go:77 # 0x1035bee k8s.io/client-go/rest/watch.(*Decoder).Decode+0x4e k8s.io/client-go@v0.27.4/rest/watch/decoder.go:49 # 0xd2a53b k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive+0xdb k8s.io/apimachinery@v0.27.4/pkg/watch/streamwatcher.go:105 7636 1 @ 0x950345 0x95031e 0x950a11 0x9508de 0x9505e7 0x951c52 0x4e4f5a 0x951bcc 0x951ba3 0x993407 0x993c66 0x99434e 0x9a651a 0x9a57a5 0x9a41c4 0x9a5425 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23f8f5e 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 # 0x950344 sync.(*Mutex).Lock+0x44 sync/mutex.go:90 # 0x95031d google.golang.org/grpc/internal/transport.(*recvBuffer).load+0x1d google.golang.org/grpc@v1.51.0/internal/transport/transport.go:126 # 0x950a10 google.golang.org/grpc/internal/transport.(*recvBufferReader).readAdditional+0x50 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:219 # 0x9508dd google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient+0xdd google.golang.org/grpc@v1.51.0/internal/transport/transport.go:214 # 0x9505e6 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read+0x146 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:175 # 0x951c51 google.golang.org/grpc/internal/transport.(*transportReader).Read+0x31 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:495 # 0x4e4f59 io.ReadAtLeast+0x99 io/io.go:332 # 0x951bcb io.ReadFull+0xab io/io.go:351 # 0x951ba2 google.golang.org/grpc/internal/transport.(*Stream).Read+0x82 google.golang.org/grpc@v1.51.0/internal/transport/transport.go:479 # 0x993406 google.golang.org/grpc.(*parser).recvMsg+0x46 google.golang.org/grpc@v1.51.0/rpc_util.go:560 # 0x993c65 google.golang.org/grpc.recvAndDecompress+0x65 google.golang.org/grpc@v1.51.0/rpc_util.go:691 # 0x99434d google.golang.org/grpc.recv+0x6d google.golang.org/grpc@v1.51.0/rpc_util.go:757 # 0x9a6519 google.golang.org/grpc.(*csAttempt).recvMsg+0x2b9 google.golang.org/grpc@v1.51.0/stream.go:1041 # 0x9a57a4 google.golang.org/grpc.(*clientStream).RecvMsg.func1+0x24 google.golang.org/grpc@v1.51.0/stream.go:881 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a5424 google.golang.org/grpc.(*clientStream).RecvMsg+0x124 google.golang.org/grpc@v1.51.0/stream.go:880 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23f8f5d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x37d github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:119 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 7408 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab177e 0x1b2440f 0x1b24645 0x1f1787d 0x23f27a6 0x23f1c8c 0x240c779 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab177d k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartRecordingToSink+0x15d k8s.io/client-go@v0.27.4/tools/record/event.go:198 # 0x1b2440e github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x26e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:107 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f1787c github.com/openshift/library-go/pkg/operator/configobserver.NewNestedConfigObserver+0x35c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:114 # 0x23f27a5 github.com/openshift/library-go/pkg/operator/configobserver.NewConfigObserver+0x11a5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:67 # 0x23f1c8b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller.NewConfigObserver+0x68b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller/observe_config_controller.go:59 # 0x240c778 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xad8 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:188 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 7336 1 @ 0x91da45 0x91da19 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x189e99a 0x23bde58 0x23c25f4 0x23c05fb 0x23c699e 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23bde57 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).getStaticPodState+0xf7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:212 # 0x23c25f3 github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).newNodeStateForInstallInProgress+0x2d3 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:687 # 0x23c05fa github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.(*InstallerController).manageInstallationPods+0xe9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:486 # 0x23c699d github.com/openshift/library-go/pkg/operator/staticpod/controller/installer.InstallerController.Sync+0x2dd github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installer/installer_controller.go:1065 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 6980 1 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189ebf0 0x23c809a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189ebef k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:99 # 0x23c8099 github.com/openshift/library-go/pkg/operator/staticpod/controller/installerstate.(*InstallerStateController).sync+0x199 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/installerstate/installer_state_controller.go:72 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 6750 1 @ 0x4a4271 0x4a4251 0x4a519f 0x54c945 0x54daeb 0x5514c5 0x475761 # 0x4a4270 sync.(*Mutex).Lock+0x90 sync/mutex.go:90 # 0x4a4250 context.propagateCancel+0x70 context/context.go:319 # 0x4a519e context.WithDeadline+0x17e context/context.go:506 # 0x54c944 net.(*Resolver).exchange+0x324 net/dnsclient_unix.go:174 # 0x54daea net.(*Resolver).tryOneName+0x46a net/dnsclient_unix.go:277 # 0x5514c4 net.(*Resolver).goLookupIPCNAMEOrder.func3.1+0x84 net/dnsclient_unix.go:653 6572 1 @ 0x91da45 0x91da19 0x57c57f 0x57cd4c 0x659de2 0x658b05 0x65815c 0x65825e 0x65bf1b 0x9205e5 0x4e5ca3 0x10602f4 0x105fe85 0x105f95e 0x105f93f 0x105f685 0x105fce9 0x18a6f50 0x1a4a745 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x91da44 sync.(*Mutex).Lock+0x224 sync/mutex.go:90 # 0x91da18 golang.org/x/net/http2.transportResponseBody.Read+0x1f8 golang.org/x/net@v0.12.0/http2/transport.go:2557 # 0x57c57e bufio.(*Reader).fill+0xfe bufio/bufio.go:106 # 0x57cd4b bufio.(*Reader).ReadByte+0x2b bufio/bufio.go:265 # 0x659de1 compress/flate.(*decompressor).huffSym+0x101 compress/flate/inflate.go:719 # 0x658b04 compress/flate.(*decompressor).huffmanBlock+0x44 compress/flate/inflate.go:494 # 0x65815b compress/flate.(*decompressor).nextBlock+0x19b compress/flate/inflate.go:327 # 0x65825d compress/flate.(*decompressor).Read+0x7d compress/flate/inflate.go:347 # 0x65bf1a compress/gzip.(*Reader).Read+0xba compress/gzip/gunzip.go:252 # 0x9205e4 golang.org/x/net/http2.(*gzipReader).Read+0xe4 golang.org/x/net@v0.12.0/http2/transport.go:3072 # 0x4e5ca2 io.ReadAll+0x102 io/io.go:701 # 0x10602f3 k8s.io/client-go/rest.(*Request).transformResponse+0xb3 k8s.io/client-go@v0.27.4/rest/request.go:1074 # 0x105fe84 k8s.io/client-go/rest.(*Request).Do.func1+0x44 k8s.io/client-go@v0.27.4/rest/request.go:1040 # 0x105f95d k8s.io/client-go/rest.(*Request).request.func3.1+0xdd k8s.io/client-go@v0.27.4/rest/request.go:1015 # 0x105f93e k8s.io/client-go/rest.(*Request).request.func3+0xbe k8s.io/client-go@v0.27.4/rest/request.go:1022 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18a6f4f k8s.io/client-go/kubernetes/typed/core/v1.(*secrets).List+0x16f k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/secret.go:95 # 0x1a4a744 k8s.io/client-go/informers/core/v1.NewFilteredSecretInformer.func1+0x144 k8s.io/client-go@v0.27.4/informers/core/v1/secret.go:65 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 6548 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1f14547 0x2407776 0x240c6e5 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f14546 github.com/openshift/library-go/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x1a6 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:73 # 0x2407775 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller.NewResourceSyncController+0x215 github.com/openshift/cluster-etcd-operator/pkg/operator/resourcesynccontroller/resourcesynccontroller.go:19 # 0x240c6e4 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xa44 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:178 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 6418 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1f1787d 0x23f27a6 0x23f1c8c 0x240c779 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1f1787c github.com/openshift/library-go/pkg/operator/configobserver.NewNestedConfigObserver+0x35c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:114 # 0x23f27a5 github.com/openshift/library-go/pkg/operator/configobserver.NewConfigObserver+0x11a5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:67 # 0x23f1c8b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller.NewConfigObserver+0x68b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller/observe_config_controller.go:59 # 0x240c778 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xad8 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:188 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 5930 2 @ 0x92c872 0x92c848 0x93b038 0x9a32ca 0x9a27da 0x9a41c4 0x9a2236 0x9a153f 0x9a0fee 0x9851fd 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff408c 0x1ffa03f 0x1ff9ebf 0x475761 # 0x92c871 sync.(*Mutex).Lock+0x51 sync/mutex.go:90 # 0x92c847 google.golang.org/grpc/internal/transport.(*controlBuffer).executeAndPut+0x27 google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:331 # 0x93b037 google.golang.org/grpc/internal/transport.(*http2Client).NewStream+0x437 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:803 # 0x9a32c9 google.golang.org/grpc.(*csAttempt).newStream+0x49 google.golang.org/grpc@v1.51.0/stream.go:458 # 0x9a27d9 google.golang.org/grpc.newClientStreamWithParams.func2+0x39 google.golang.org/grpc@v1.51.0/stream.go:327 # 0x9a41c3 google.golang.org/grpc.(*clientStream).withRetry+0x143 google.golang.org/grpc@v1.51.0/stream.go:732 # 0x9a2235 google.golang.org/grpc.newClientStreamWithParams+0xc75 google.golang.org/grpc@v1.51.0/stream.go:336 # 0x9a153e google.golang.org/grpc.newClientStream.func2+0x9e google.golang.org/grpc@v1.51.0/stream.go:193 # 0x9a0fed google.golang.org/grpc.newClientStream+0x4cd google.golang.org/grpc@v1.51.0/stream.go:228 # 0x9851fc google.golang.org/grpc.invoke+0x7c google.golang.org/grpc@v1.51.0/call.go:66 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff408b github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x86b github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:141 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 5060 1 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ff5bf2 0x23fbb03 0x23fb545 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ff5bf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).MemberList+0xf1 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:272 # 0x23fbb02 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).attemptToDeleteMachineDeletionHook+0x282 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:129 # 0x23fb544 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks.(*machineDeletionHooksController).sync+0x144 github.com/openshift/cluster-etcd-operator/pkg/operator/machinedeletionhooks/machinedeletionhooks.go:97 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4864 1 @ 0x481bcb 0xd28d77 0xd28fc9 0x1ab2a08 0x1ab22e5 0x1b243bf 0x1b24645 0x1b2816a 0x1b2a065 0x1f1789c 0x23f27a6 0x23f1c8c 0x240c779 0x23aaef4 0x475761 # 0x481bca sync.(*WaitGroup).Wait+0x4a sync/waitgroup.go:116 # 0xd28d76 k8s.io/apimachinery/pkg/watch.(*Broadcaster).blockQueue+0x156 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:134 # 0xd28fc8 k8s.io/apimachinery/pkg/watch.(*Broadcaster).Watch+0x88 k8s.io/apimachinery@v0.27.4/pkg/watch/mux.go:143 # 0x1ab2a07 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher+0x27 k8s.io/client-go@v0.27.4/tools/record/event.go:314 # 0x1ab22e4 k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartLogging+0x64 k8s.io/client-go@v0.27.4/tools/record/event.go:296 # 0x1b243be github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).ForComponent+0x21e github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:106 # 0x1b24644 github.com/openshift/library-go/pkg/operator/events.(*upstreamRecorder).WithComponentSuffix+0xa4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/events/recorder_upstream.go:127 # 0x1b28169 github.com/openshift/library-go/pkg/controller/factory.NewSyncContext+0x229 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/controller_context.go:31 # 0x1b2a064 github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x84 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:252 # 0x1f1789b github.com/openshift/library-go/pkg/operator/configobserver.NewNestedConfigObserver+0x37b github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:114 # 0x23f27a5 github.com/openshift/library-go/pkg/operator/configobserver.NewConfigObserver+0x11a5 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:67 # 0x23f1c8b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller.NewConfigObserver+0x68b github.com/openshift/cluster-etcd-operator/pkg/operator/configobservation/configobservercontroller/observe_config_controller.go:59 # 0x240c778 github.com/openshift/cluster-etcd-operator/pkg/operator.RunOperator+0xad8 github.com/openshift/cluster-etcd-operator/pkg/operator/starter.go:188 # 0x23aaef3 github.com/openshift/library-go/pkg/controller/controllercmd.ControllerBuilder.getOnStartedLeadingFunc.func1.1+0x73 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:351 4612 1 @ 0x1a84553 0x1a84524 0x1a44b25 0x1a44abe 0x1a9c265 0x23e2330 0x23e1d85 0x23e380c 0x23f5d1f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x1a84552 sync.(*Mutex).Lock+0x92 sync/mutex.go:90 # 0x1a84523 k8s.io/client-go/informers.(*sharedInformerFactory).InformerFor+0x63 k8s.io/client-go@v0.27.4/informers/factory.go:190 # 0x1a44b24 k8s.io/client-go/informers/core/v1.(*configMapInformer).Informer+0x84 k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:85 # 0x1a44abd k8s.io/client-go/informers/core/v1.(*configMapInformer).Lister+0x1d k8s.io/client-go@v0.27.4/informers/core/v1/configmap.go:89 # 0x1a9c264 github.com/openshift/library-go/pkg/operator/v1helpers.configMapLister.ConfigMaps+0x64 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/informers.go:84 # 0x23e232f github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.IsBootstrapComplete+0x8f github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:168 # 0x23e1d84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.CheckSafeToScaleCluster+0x84 github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/bootstrap.go:111 # 0x23e380b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers.(*QuorumCheck).IsSafeToUpdateRevision+0x4b github.com/openshift/cluster-etcd-operator/pkg/operator/ceohelpers/qourum_check.go:37 # 0x23f5d1e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner.(*EtcdCertSignerController).sync+0x5e github.com/openshift/cluster-etcd-operator/pkg/operator/etcdcertsigner/etcdcertsignercontroller.go:118 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 4526 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308cf9 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308cf8 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x278 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:149 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 4202 1 @ 0x107e7dd 0x107e7b2 0x107d74d 0xc086db 0xc088d7 0xc08a72 0xc0783a 0xc07629 0xc0759f 0x107d648 0x1b255da 0x1b2587d 0x1f14cd3 0x475761 # 0x107e7dc sync.(*Mutex).Lock+0x5c sync/mutex.go:90 # 0x107e7b1 k8s.io/client-go/tools/cache.(*sharedIndexInformer).HasSynced+0x31 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:513 # 0x107d74c k8s.io/client-go/tools/cache.WaitForCacheSync.func1+0x4c k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:329 # 0xc086da k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1+0x1a k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:109 # 0xc088d6 k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext+0x56 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:154 # 0xc08a71 k8s.io/apimachinery/pkg/util/wait.waitForWithContext+0x111 k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:207 # 0xc07839 k8s.io/apimachinery/pkg/util/wait.poll+0x99 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:260 # 0xc07628 k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext+0x48 k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:200 # 0xc0759e k8s.io/apimachinery/pkg/util/wait.PollImmediateUntil+0x3e k8s.io/apimachinery@v0.27.4/pkg/util/wait/poll.go:187 # 0x107d647 k8s.io/client-go/tools/cache.WaitForCacheSync+0x87 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:326 # 0x1b255d9 github.com/openshift/library-go/pkg/controller/factory.waitForNamedCacheSync+0x119 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:69 # 0x1b2587c github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run+0x11c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:85 # 0x1f14cd2 github.com/openshift/library-go/pkg/operator/resourcesynccontroller.(*ResourceSyncController).Run+0x32 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:95 3974 1 @ 0x4809ff 0x4809da 0x48094e 0x480685 0x7bcd1f 0x7be5b1 0x94ea53 0x92d46e 0x936d0a 0x475761 # 0x4809fe sync.(*Mutex).Lock+0x7e sync/mutex.go:90 # 0x4809d9 sync.(*Pool).pinSlow+0x59 sync/pool.go:216 # 0x48094d sync.(*Pool).pin+0x4d sync/pool.go:209 # 0x480684 sync.(*Pool).Get+0x24 sync/pool.go:131 # 0x7bcd1e crypto/tls.(*Conn).writeRecordLocked+0x7e crypto/tls/conn.go:950 # 0x7be5b0 crypto/tls.(*Conn).Write+0x410 crypto/tls/conn.go:1186 # 0x94ea52 google.golang.org/grpc/internal/transport.(*bufWriter).Flush+0x52 google.golang.org/grpc@v1.51.0/internal/transport/http_util.go:354 # 0x92d46d google.golang.org/grpc/internal/transport.(*loopyWriter).run+0x6d google.golang.org/grpc@v1.51.0/internal/transport/controlbuf.go:575 # 0x936d09 google.golang.org/grpc/internal/transport.newHTTP2Client.func6+0x69 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:442 3846 1 @ 0x451705 0x91e052 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x1f00885 0x23cf085 0x23cf038 0x23ce497 0x23cf67a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x451704 runtime.selectgo+0x404 runtime/select.go:508 # 0x91e051 golang.org/x/net/http2.transportResponseBody.Close+0x231 golang.org/x/net@v0.12.0/http2/transport.go:2606 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x1f00884 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:191 # 0x23cf084 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPod+0x7c4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:72 # 0x23cf037 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).ensurePrunePod+0x777 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:217 # 0x23ce496 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).pruneDiskResources+0x156 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:153 # 0x23cf679 github.com/openshift/library-go/pkg/operator/staticpod/controller/prune.(*PruneController).sync+0x1b9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/prune/prune_controller.go:274 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3514 1 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x23f926a 0x23f9220 0x23f885a 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x23f9269 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x689 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x23f921f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).syncConfigMap+0x63f github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:160 # 0x23f8859 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller.(*EtcdEndpointsController).sync+0x79 github.com/openshift/cluster-etcd-operator/pkg/operator/etcdendpointscontroller/etcdendpointscontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3466 1 @ 0x5f85dc 0x5f85bb 0x60a9fc 0x60ab3e 0x607cb8 0x60803a 0x7fbd82 0x7ff837 0x98c3ef 0x98bc25 0x98a005 0x475761 # 0x5f85db sync.(*Mutex).Lock+0x9b sync/mutex.go:90 # 0x5f85ba log.(*Logger).Output+0x7a log/log.go:175 # 0x60a9fb google.golang.org/grpc/grpclog.(*loggerT).output+0x23b google.golang.org/grpc@v1.51.0/grpclog/loggerv2.go:174 # 0x60ab3d google.golang.org/grpc/grpclog.(*loggerT).Infoln+0x3d google.golang.org/grpc@v1.51.0/grpclog/loggerv2.go:191 # 0x607cb7 google.golang.org/grpc/internal/grpclog.InfoDepth+0x57 google.golang.org/grpc@v1.51.0/internal/grpclog/grpclog.go:37 # 0x608039 google.golang.org/grpc/grpclog.(*componentData).InfoDepth+0x139 google.golang.org/grpc@v1.51.0/grpclog/component.go:36 # 0x7fbd81 google.golang.org/grpc/internal/channelz.AddTraceEvent+0x301 google.golang.org/grpc@v1.51.0/internal/channelz/funcs.go:340 # 0x7ff836 google.golang.org/grpc/internal/channelz.Infof+0x96 google.golang.org/grpc@v1.51.0/internal/channelz/logging.go:43 # 0x98c3ee google.golang.org/grpc.(*addrConn).tryAllAddrs+0x42e google.golang.org/grpc@v1.51.0/clientconn.go:1211 # 0x98bc24 google.golang.org/grpc.(*addrConn).resetTransport+0x1c4 google.golang.org/grpc@v1.51.0/clientconn.go:1148 # 0x98a004 google.golang.org/grpc.(*addrConn).connect+0x104 google.golang.org/grpc@v1.51.0/clientconn.go:803 3380 1 @ 0x1084268 0x108423a 0x1082ac6 0x18ac9ad 0x1a9990b 0x1f016c5 0x1f03d25 0x1f03d26 0x1f1654d 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x1084267 sync.(*RWMutex).RLock+0x67 sync/rwmutex.go:71 # 0x1084239 k8s.io/client-go/tools/cache.(*threadSafeMap).Get+0x39 k8s.io/client-go@v0.27.4/tools/cache/thread_safe_store.go:239 # 0x1082ac5 k8s.io/client-go/tools/cache.(*cache).GetByKey+0x25 k8s.io/client-go@v0.27.4/tools/cache/store.go:237 # 0x18ac9ac k8s.io/client-go/listers/core/v1.configMapNamespaceLister.Get+0x8c k8s.io/client-go@v0.27.4/listers/core/v1/configmap.go:91 # 0x1a9990a github.com/openshift/library-go/pkg/operator/v1helpers.combinedConfigMapInterface.Get+0xca github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/core_getters.go:51 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x1f03d24 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x324 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x1f03d25 github.com/openshift/library-go/pkg/operator/resource/resourceapply.SyncPartialConfigMap+0x325 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:492 # 0x1f1654c github.com/openshift/library-go/pkg/operator/resourcesynccontroller.(*ResourceSyncController).Sync+0x68c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resourcesynccontroller/resourcesync_controller.go:218 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3284 1 @ 0x1084268 0x108423a 0x1082ac6 0x18ac9ad 0x1a9990b 0x23b50a8 0x23b479b 0x23b6b05 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x1084267 sync.(*RWMutex).RLock+0x67 sync/rwmutex.go:71 # 0x1084239 k8s.io/client-go/tools/cache.(*threadSafeMap).Get+0x39 k8s.io/client-go@v0.27.4/tools/cache/thread_safe_store.go:239 # 0x1082ac5 k8s.io/client-go/tools/cache.(*cache).GetByKey+0x25 k8s.io/client-go@v0.27.4/tools/cache/store.go:237 # 0x18ac9ac k8s.io/client-go/listers/core/v1.configMapNamespaceLister.Get+0x8c k8s.io/client-go@v0.27.4/listers/core/v1/configmap.go:91 # 0x1a9990a github.com/openshift/library-go/pkg/operator/v1helpers.combinedConfigMapInterface.Get+0xca github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/core_getters.go:51 # 0x23b50a7 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.isLatestRevisionCurrent+0x2a7 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:139 # 0x23b479a github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.createRevisionIfNeeded+0x9a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:88 # 0x23b6b04 github.com/openshift/library-go/pkg/operator/revisioncontroller.RevisionController.sync+0x2a4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/revisioncontroller/revision_controller.go:312 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3258 1 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff6479 0x23e8d22 0x23e8c70 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff6478 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).UnhealthyMembers+0x78 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:328 # 0x23e8d21 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).reconcileMembers+0x61 github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:103 # 0x23e8c6f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller.(*ClusterMemberController).sync+0x4f github.com/openshift/cluster-etcd-operator/pkg/operator/clustermembercontroller/clustermembercontroller.go:99 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3172 1 @ 0x1084268 0x108423a 0x1082ac6 0x226101a 0x2261646 0x23ae14e 0x1a9af5e 0x1a95753 0xc0880d 0xc063bf 0x1a95688 0x1a9ae66 0x1a9ad9d 0x1f18525 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x1084267 sync.(*RWMutex).RLock+0x67 sync/rwmutex.go:71 # 0x1084239 k8s.io/client-go/tools/cache.(*threadSafeMap).Get+0x39 k8s.io/client-go@v0.27.4/tools/cache/thread_safe_store.go:239 # 0x1082ac5 k8s.io/client-go/tools/cache.(*cache).GetByKey+0x25 k8s.io/client-go@v0.27.4/tools/cache/store.go:237 # 0x2261019 k8s.io/client-go/dynamic/dynamiclister.(*dynamicLister).Get+0x39 k8s.io/client-go@v0.27.4/dynamic/dynamiclister/lister.go:51 # 0x2261645 k8s.io/client-go/dynamic/dynamiclister.(*dynamicListerShim).Get+0x25 k8s.io/client-go@v0.27.4/dynamic/dynamiclister/shim.go:55 # 0x23ae14d github.com/openshift/library-go/pkg/operator/genericoperatorclient.dynamicOperatorClient.GetOperatorState+0x6d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/genericoperatorclient/dynamic_operator_client.go:86 # 0x1a9af5d github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus.func1+0x7d github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:163 # 0x1a95752 k8s.io/client-go/util/retry.OnError.func1+0x32 k8s.io/client-go@v0.27.4/util/retry/util.go:51 # 0xc0880c k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection+0x4c k8s.io/apimachinery@v0.27.4/pkg/util/wait/wait.go:145 # 0xc063be k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff+0x5e k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:461 # 0x1a95687 k8s.io/client-go/util/retry.OnError+0xa7 k8s.io/client-go@v0.27.4/util/retry/util.go:50 # 0x1a9ae65 k8s.io/client-go/util/retry.RetryOnConflict+0x125 k8s.io/client-go@v0.27.4/util/retry/util.go:104 # 0x1a9ad9c github.com/openshift/library-go/pkg/operator/v1helpers.UpdateStatus+0x5c github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/v1helpers/helpers.go:162 # 0x1f18524 github.com/openshift/library-go/pkg/operator/configobserver.ConfigObserver.sync+0xc04 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/configobserver/config_observer_controller.go:177 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 3134 1 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x188f9da 0x1f016c5 0x24095a5 0x240952c 0x2408e6b 0x2408a7f 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x188f9d9 k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/configmap.go:78 # 0x1f016c4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMapImproved+0xc4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:265 # 0x24095a4 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyConfigMap+0x504 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/core.go:82 # 0x240952b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).manageScriptConfigMap+0x48b github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:132 # 0x2408e6a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.(*ScriptController).createScriptConfigMap+0x4a github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:103 # 0x2408a7e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller.ScriptController.sync+0x9e github.com/openshift/cluster-etcd-operator/pkg/operator/scriptcontroller/scriptcontroller.go:71 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2982 1 @ 0x8f4eca 0x8f4e9f 0x8f6545 0x911f54 0x92095b 0x920955 0x8cf42a 0x8b6799 0x102c4e5 0x102b1f0 0x873e57 0x8736db 0x8759db 0x105f545 0x105f39e 0x105fce9 0x19b235a 0x1f0c295 0x23bab9f 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x8f4ec9 sync.(*Mutex).Lock+0x129 sync/mutex.go:90 # 0x8f4e9e golang.org/x/net/http2.(*clientConnPool).getClientConn+0xfe golang.org/x/net@v0.12.0/http2/client_conn_pool.go:76 # 0x8f6544 golang.org/x/net/http2.noDialClientConnPool.GetClientConn+0x24 golang.org/x/net@v0.12.0/http2/client_conn_pool.go:285 # 0x911f53 golang.org/x/net/http2.(*Transport).RoundTripOpt+0x173 golang.org/x/net@v0.12.0/http2/transport.go:554 # 0x92095a golang.org/x/net/http2.(*Transport).RoundTrip+0x1a golang.org/x/net@v0.12.0/http2/transport.go:513 # 0x920954 golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip+0x14 golang.org/x/net@v0.12.0/http2/transport.go:3112 # 0x8cf429 net/http.(*Transport).roundTrip+0x3c9 net/http/transport.go:548 # 0x8b6798 net/http.(*Transport).RoundTrip+0x18 net/http/roundtrip.go:17 # 0x102c4e4 k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip+0x3e4 k8s.io/client-go@v0.27.4/transport/round_trippers.go:317 # 0x102b1ef k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip+0x34f k8s.io/client-go@v0.27.4/transport/round_trippers.go:168 # 0x873e56 net/http.send+0x5f6 net/http/client.go:252 # 0x8736da net/http.(*Client).send+0x9a net/http/client.go:176 # 0x8759da net/http.(*Client).do+0x8fa net/http/client.go:716 # 0x105f544 net/http.(*Client).Do+0x3a4 net/http/client.go:582 # 0x105f39d k8s.io/client-go/rest.(*Request).request+0x1fd k8s.io/client-go@v0.27.4/rest/request.go:999 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x19b2359 k8s.io/client-go/kubernetes/typed/policy/v1.(*podDisruptionBudgets).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/policy/v1/poddisruptionbudget.go:80 # 0x1f0c294 github.com/openshift/library-go/pkg/operator/resource/resourceapply.ApplyPodDisruptionBudget+0xb4 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/resource/resourceapply/policy.go:18 # 0x23bab9e github.com/openshift/library-go/pkg/operator/staticpod/controller/guard.(*GuardController).sync+0x6fe github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/guard/guard_controller.go:233 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2832 1 @ 0x40b61d 0x107f652 0x107f689 0x107ed87 0x1b2a9ac 0x1f94ff7 0x475761 # 0x40b61c runtime.chansend1+0x1c runtime/chan.go:145 # 0x107f651 k8s.io/client-go/tools/cache.(*processorListener).add+0x671 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:928 # 0x107f688 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandlerWithResyncPeriod+0x6a8 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:628 # 0x107ed86 k8s.io/client-go/tools/cache.(*sharedIndexInformer).AddEventHandler+0x26 k8s.io/client-go@v0.27.4/tools/cache/shared_informer.go:556 # 0x1b2a9ab github.com/openshift/library-go/pkg/controller/factory.(*Factory).ToController+0x9cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/factory.go:294 # 0x1f94ff6 github.com/openshift/library-go/pkg/operator/status.(*StatusSyncer).Run+0x216 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/status/status_controller.go:102 2832 1 @ 0x5f85dc 0x5f85bb 0x60a9fc 0x60ab3e 0x607cb8 0x60803a 0x7fbd82 0x7ff837 0x98b94b 0x98db90 0x98d4e8 0x98cb8c 0x98c485 0x98bc25 0x98a005 0x475761 # 0x5f85db sync.(*Mutex).Lock+0x9b sync/mutex.go:90 # 0x5f85ba log.(*Logger).Output+0x7a log/log.go:175 # 0x60a9fb google.golang.org/grpc/grpclog.(*loggerT).output+0x23b google.golang.org/grpc@v1.51.0/grpclog/loggerv2.go:174 # 0x60ab3d google.golang.org/grpc/grpclog.(*loggerT).Infoln+0x3d google.golang.org/grpc@v1.51.0/grpclog/loggerv2.go:191 # 0x607cb7 google.golang.org/grpc/internal/grpclog.InfoDepth+0x57 google.golang.org/grpc@v1.51.0/internal/grpclog/grpclog.go:37 # 0x608039 google.golang.org/grpc/grpclog.(*componentData).InfoDepth+0x139 google.golang.org/grpc@v1.51.0/grpclog/component.go:36 # 0x7fbd81 google.golang.org/grpc/internal/channelz.AddTraceEvent+0x301 google.golang.org/grpc@v1.51.0/internal/channelz/funcs.go:340 # 0x7ff836 google.golang.org/grpc/internal/channelz.Infof+0x96 google.golang.org/grpc@v1.51.0/internal/channelz/logging.go:43 # 0x98b94a google.golang.org/grpc.(*addrConn).updateConnectivityState+0xaa google.golang.org/grpc@v1.51.0/clientconn.go:1100 # 0x98db8f google.golang.org/grpc.(*addrConn).startHealthCheck.func1+0x2f google.golang.org/grpc@v1.51.0/clientconn.go:1323 # 0x98d4e7 google.golang.org/grpc.(*addrConn).startHealthCheck+0x3a7 google.golang.org/grpc@v1.51.0/clientconn.go:1332 # 0x98cb8b google.golang.org/grpc.(*addrConn).createTransport+0x62b google.golang.org/grpc@v1.51.0/clientconn.go:1303 # 0x98c484 google.golang.org/grpc.(*addrConn).tryAllAddrs+0x4c4 google.golang.org/grpc@v1.51.0/clientconn.go:1213 # 0x98bc24 google.golang.org/grpc.(*addrConn).resetTransport+0x1c4 google.golang.org/grpc@v1.51.0/clientconn.go:1148 # 0x98a004 google.golang.org/grpc.(*addrConn).connect+0x104 google.golang.org/grpc@v1.51.0/clientconn.go:803 2766 1 @ 0x4809ff 0x4809da 0x48094e 0x480685 0x501b65 0x502136 0x23df714 0x23dee55 0x23de0b8 0x23ddce5 0x23de8ef 0x23e0f8b 0xc05afe 0xc059b6 0xc058a9 0xc05765 0x475761 # 0x4809fe sync.(*Mutex).Lock+0x7e sync/mutex.go:90 # 0x4809d9 sync.(*Pool).pinSlow+0x59 sync/pool.go:216 # 0x48094d sync.(*Pool).pin+0x4d sync/pool.go:209 # 0x480684 sync.(*Pool).Get+0x24 sync/pool.go:131 # 0x501b64 fmt.newPrinter+0x24 fmt/print.go:152 # 0x502135 fmt.Sprintf+0x35 fmt/print.go:238 # 0x23df713 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.getEscapedIPAddress+0x1b3 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/etcd_env.go:174 # 0x23dee54 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.getEtcdEnvVars+0xd4 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/etcd_env.go:92 # 0x23de0b7 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).checkEnvVars+0x177 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:138 # 0x23ddce4 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).sync+0x44 github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:109 # 0x23de8ee github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).processNextWorkItem+0x10e github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:204 # 0x23e0f8a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar.(*EnvVarController).runWorker+0x4a github.com/openshift/cluster-etcd-operator/pkg/etcdenvvar/envvarcontroller.go:193 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05764 k8s.io/apimachinery/pkg/util/wait.Until+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:161 2534 1 @ 0x8471f7 0x8471d0 0x989545 0x98948d 0x98b976 0x989fd4 0x475761 # 0x8471f6 sync.(*Mutex).Lock+0x56 sync/mutex.go:90 # 0x8471cf google.golang.org/grpc/internal/buffer.(*Unbounded).Put+0x2f google.golang.org/grpc@v1.51.0/internal/buffer/unbounded.go:49 # 0x989544 google.golang.org/grpc.(*ccBalancerWrapper).updateSubConnState+0xe4 google.golang.org/grpc@v1.51.0/balancer_conn_wrappers.go:208 # 0x98948c google.golang.org/grpc.(*ClientConn).handleSubConnStateChange+0x2c google.golang.org/grpc@v1.51.0/clientconn.go:696 # 0x98b975 google.golang.org/grpc.(*addrConn).updateConnectivityState+0xd5 google.golang.org/grpc@v1.51.0/clientconn.go:1101 # 0x989fd3 google.golang.org/grpc.(*addrConn).connect+0xd3 google.golang.org/grpc@v1.51.0/clientconn.go:800 2428 1 @ 0x5f85dc 0x5f85bb 0x60a9fc 0x60ab3e 0x607cb8 0x60803a 0x7fbd82 0x7ff837 0x99319b 0x992232 0xb84150 0xb83fb8 0xb83d59 0x991dff 0x986848 0xb8a897 0xb8a2fa 0xb8b835 0x1ff3ffd 0x1ff3f9d 0x1ffa03f 0x1ff9ebf 0x475761 # 0x5f85db sync.(*Mutex).Lock+0x9b sync/mutex.go:90 # 0x5f85ba log.(*Logger).Output+0x7a log/log.go:175 # 0x60a9fb google.golang.org/grpc/grpclog.(*loggerT).output+0x23b google.golang.org/grpc@v1.51.0/grpclog/loggerv2.go:174 # 0x60ab3d google.golang.org/grpc/grpclog.(*loggerT).Infoln+0x3d google.golang.org/grpc@v1.51.0/grpclog/loggerv2.go:191 # 0x607cb7 google.golang.org/grpc/internal/grpclog.InfoDepth+0x57 google.golang.org/grpc@v1.51.0/internal/grpclog/grpclog.go:37 # 0x608039 google.golang.org/grpc/grpclog.(*componentData).InfoDepth+0x139 google.golang.org/grpc@v1.51.0/grpclog/component.go:36 # 0x7fbd81 google.golang.org/grpc/internal/channelz.AddTraceEvent+0x301 google.golang.org/grpc@v1.51.0/internal/channelz/funcs.go:340 # 0x7ff836 google.golang.org/grpc/internal/channelz.Infof+0x96 google.golang.org/grpc@v1.51.0/internal/channelz/logging.go:43 # 0x99319a google.golang.org/grpc.(*ccResolverWrapper).addChannelzTraceEvent+0x33a google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:175 # 0x992231 google.golang.org/grpc.(*ccResolverWrapper).UpdateState+0x111 google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:100 # 0xb8414f google.golang.org/grpc/resolver/manual.(*Resolver).UpdateState+0x24f google.golang.org/grpc@v1.51.0/resolver/manual/manual.go:90 # 0xb83fb7 go.etcd.io/etcd/client/v3/internal/resolver.EtcdManualResolver.updateState+0xb7 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:72 # 0xb83d58 go.etcd.io/etcd/client/v3/internal/resolver.(*EtcdManualResolver).Build+0x1b8 go.etcd.io/etcd/client/v3@v3.5.7/internal/resolver/resolver.go:52 # 0x991dfe google.golang.org/grpc.newCCResolverWrapper+0x2be google.golang.org/grpc@v1.51.0/resolver_conn_wrapper.go:72 # 0x986847 google.golang.org/grpc.DialContext+0x1407 google.golang.org/grpc@v1.51.0/clientconn.go:296 # 0xb8a896 go.etcd.io/etcd/client/v3.(*Client).dial+0x416 go.etcd.io/etcd/client/v3@v3.5.7/client.go:302 # 0xb8a2f9 go.etcd.io/etcd/client/v3.(*Client).dialWithBalancer+0x1b9 go.etcd.io/etcd/client/v3@v3.5.7/client.go:280 # 0xb8b834 go.etcd.io/etcd/client/v3.newClient+0xa54 go.etcd.io/etcd/client/v3@v3.5.7/client.go:413 # 0x1ff3ffc go.etcd.io/etcd/client/v3.New+0x7dc go.etcd.io/etcd/client/v3@v3.5.7/client.go:81 # 0x1ff3f9c github.com/openshift/cluster-etcd-operator/pkg/etcdcli.newEtcdClientWithClientOpts+0x77c github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:127 # 0x1ffa03e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.checkSingleMemberHealth+0xde github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:102 # 0x1ff9ebe github.com/openshift/cluster-etcd-operator/pkg/etcdcli.getMemberHealth.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/health.go:58 2272 1 @ 0x40c378 0x93c0f7 0x93be6b 0x9a6c26 0x9a5a92 0x9a54f4 0x985257 0x1f98ad7 0x9876f4 0xb994b0 0x9873e8 0x9850a3 0xb22ae9 0xb97143 0xb8d784 0x1ffbf7f 0x1ff8fa9 0x1ff5f46 0x23f36db 0x23f2e27 0x1f8f786 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 # 0x40c377 runtime.chanrecv1+0x17 runtime/chan.go:442 # 0x93c0f6 google.golang.org/grpc/internal/transport.(*http2Client).closeStream+0x236 google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:871 # 0x93be6a google.golang.org/grpc/internal/transport.(*http2Client).CloseStream+0x6a google.golang.org/grpc@v1.51.0/internal/transport/http2_client.go:863 # 0x9a6c25 google.golang.org/grpc.(*csAttempt).finish+0x125 google.golang.org/grpc@v1.51.0/stream.go:1102 # 0x9a5a91 google.golang.org/grpc.(*clientStream).finish+0x151 google.golang.org/grpc@v1.51.0/stream.go:957 # 0x9a54f3 google.golang.org/grpc.(*clientStream).RecvMsg+0x1f3 google.golang.org/grpc@v1.51.0/stream.go:894 # 0x985256 google.golang.org/grpc.invoke+0xd6 google.golang.org/grpc@v1.51.0/call.go:73 # 0x1f98ad6 github.com/grpc-ecosystem/go-grpc-prometheus.(*ClientMetrics).UnaryClientInterceptor.func1+0x116 github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/client_metrics.go:112 # 0x9876f3 google.golang.org/grpc.getChainUnaryInvoker.func1+0x153 google.golang.org/grpc@v1.51.0/clientconn.go:361 # 0xb994af go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1+0x5ef go.etcd.io/etcd/client/v3@v3.5.7/retry_interceptor.go:58 # 0x9873e7 google.golang.org/grpc.chainUnaryClientInterceptors.func1+0x127 google.golang.org/grpc@v1.51.0/clientconn.go:349 # 0x9850a2 google.golang.org/grpc.(*ClientConn).Invoke+0x222 google.golang.org/grpc@v1.51.0/call.go:35 # 0xb22ae8 go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberList+0xc8 go.etcd.io/etcd/api/v3@v3.5.7/etcdserverpb/rpc.pb.go:7101 # 0xb97142 go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberList+0x122 go.etcd.io/etcd/client/v3@v3.5.7/retry.go:167 # 0xb8d783 go.etcd.io/etcd/client/v3.(*cluster).MemberList+0x63 go.etcd.io/etcd/client/v3@v3.5.7/cluster.go:127 # 0x1ffbf7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli.NewDefaultEtcdClientPool.func1+0x7e github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:143 # 0x1ff8fa8 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*EtcdClientPool).Get+0x408 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli_pool.go:92 # 0x1ff5f45 github.com/openshift/cluster-etcd-operator/pkg/etcdcli.(*etcdClientGetter).Status+0x65 github.com/openshift/cluster-etcd-operator/pkg/etcdcli/etcdcli.go:290 # 0x23f36da github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).runDefrag+0x39a github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:149 # 0x23f2e26 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller.(*DefragController).sync+0x86 github.com/openshift/cluster-etcd-operator/pkg/operator/defragcontroller/defragcontroller.go:87 # 0x1f8f785 github.com/openshift/cluster-etcd-operator/pkg/operator/health.(*CheckingSyncWrapper).Sync+0x45 github.com/openshift/cluster-etcd-operator/pkg/operator/health/checking_sync_wrapper.go:22 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 2076 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x23a980b 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x23a980a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0x86a github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:259 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 2010 1 @ 0x911306 0x9112d4 0x91de93 0x1064a44 0x105f972 0x105f685 0x105fce9 0x189e99a 0x23d275a 0x1b270e7 0x1b2792c 0x1b26fb6 0xc05c65 0xc05afe 0xc059b6 0xc058a9 0xc05bd9 0x1b26ed3 0x1b26eb6 0x1b26a92 0x475761 # 0x911305 sync.(*Mutex).Lock+0x65 sync/mutex.go:90 # 0x9112d3 golang.org/x/net/http2.(*clientStream).abortStream+0x33 golang.org/x/net@v0.12.0/http2/transport.go:412 # 0x91de92 golang.org/x/net/http2.transportResponseBody.Close+0x72 golang.org/x/net@v0.12.0/http2/transport.go:2586 # 0x1064a43 k8s.io/client-go/rest.readAndCloseResponseBody+0x123 k8s.io/client-go@v0.27.4/rest/with_retry.go:354 # 0x105f971 k8s.io/client-go/rest.(*Request).request.func3+0xf1 k8s.io/client-go@v0.27.4/rest/request.go:1023 # 0x105f684 k8s.io/client-go/rest.(*Request).request+0x4e4 k8s.io/client-go@v0.27.4/rest/request.go:1024 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x189e999 k8s.io/client-go/kubernetes/typed/core/v1.(*pods).Get+0x159 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/pod.go:82 # 0x23d2759 github.com/openshift/library-go/pkg/operator/staticpod/controller/staticpodstate.(*StaticPodStateController).sync+0x3f9 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/operator/staticpod/controller/staticpodstate/staticpodstate_controller.go:86 # 0x1b270e6 github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile+0x46 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:201 # 0x1b2792b github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem+0x1cb github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:260 # 0x1b26fb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1+0x95 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:192 # 0xc05c64 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1+0x24 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0xc05afd k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1+0x3d k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:226 # 0xc059b5 k8s.io/apimachinery/pkg/util/wait.BackoffUntil+0xb5 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:227 # 0xc058a8 k8s.io/apimachinery/pkg/util/wait.JitterUntil+0x88 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:204 # 0xc05bd8 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext+0x98 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:259 # 0x1b26ed2 k8s.io/apimachinery/pkg/util/wait.UntilWithContext+0x52 k8s.io/apimachinery@v0.27.4/pkg/util/wait/backoff.go:170 # 0x1b26eb5 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker+0x35 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:183 # 0x1b26a91 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2+0x71 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/factory/base_controller.go:117 1776 1 @ 0x101b548 0x101b51c 0x101ac6e 0x101d1cb 0x101d1bf 0x105cd1a 0x105f3bb 0x105f3a3 0x105fce9 0x18992b9 0x1a476e7 0x1075935 0x1079cd5 0x106e555 0x106e6e6 0x1079aae 0x475761 # 0x101b547 sync.(*Mutex).Lock+0xa7 sync/mutex.go:90 # 0x101b51b golang.org/x/time/rate.(*Limiter).reserveN+0x7b golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:308 # 0x101ac6d golang.org/x/time/rate.(*Limiter).WaitN+0x24d golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:245 # 0x101d1ca golang.org/x/time/rate.(*Limiter).Wait+0x2a golang.org/x/time@v0.0.0-20220210224613-90d013bbcef8/rate/rate.go:216 # 0x101d1be k8s.io/client-go/util/flowcontrol.(*tokenBucketRateLimiter).Wait+0x1e k8s.io/client-go@v0.27.4/util/flowcontrol/throttle.go:131 # 0x105cd19 k8s.io/client-go/rest.(*Request).tryThrottleWithInfo+0xb9 k8s.io/client-go@v0.27.4/rest/request.go:613 # 0x105f3ba k8s.io/client-go/rest.(*Request).tryThrottle+0x21a k8s.io/client-go@v0.27.4/rest/request.go:641 # 0x105f3a2 k8s.io/client-go/rest.(*Request).request+0x202 k8s.io/client-go@v0.27.4/rest/request.go:965 # 0x105fce8 k8s.io/client-go/rest.(*Request).Do+0xc8 k8s.io/client-go@v0.27.4/rest/request.go:1039 # 0x18992b8 k8s.io/client-go/kubernetes/typed/core/v1.(*nodes).List+0x158 k8s.io/client-go@v0.27.4/kubernetes/typed/core/v1/node.go:93 # 0x1a476e6 k8s.io/client-go/informers/core/v1.NewFilteredNodeInformer.func1+0x126 k8s.io/client-go@v0.27.4/informers/core/v1/node.go:64 # 0x1075934 k8s.io/client-go/tools/cache.(*ListWatch).List+0x54 k8s.io/client-go@v0.27.4/tools/cache/listwatch.go:106 # 0x1079cd4 k8s.io/client-go/tools/cache.(*Reflector).list.func1.2+0x54 k8s.io/client-go@v0.27.4/tools/cache/reflector.go:486 # 0x106e554 k8s.io/client-go/tools/pager.SimplePageFunc.func1+0x54 k8s.io/client-go@v0.27.4/tools/pager/pager.go:40 # 0x106e6e5 k8s.io/client-go/tools/pager.(*ListPager).List+0x165 k8s.io/client-go@v0.27.4/tools/pager/pager.go:92 # 0x1079aad k8s.io/client-go/tools/cache.(*Reflector).list.func1+0x1ed k8s.io/client-go@v0.27.4/tools/cache/reflector.go:511 1620 1 @ 0x40c3b8 0x1adeaf0 0x1af4683 0x1b216e7 0x233be85 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1af4682 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentMetricHandler+0x1c2 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:254 # 0x1b216e6 k8s.io/component-base/metrics/legacyregistry.HandlerWithReset+0x66 k8s.io/component-base@v0.27.4/metrics/legacyregistry/registry.go:66 # 0x233be84 k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:44 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 1594 1 @ 0x40c3b8 0x1adeaf0 0x1b11497 0x2324d95 0x232539d 0x480462 0x232531e 0x23252df 0x237f5c5 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1b11496 k8s.io/component-base/metrics.(*kubeRegistry).Register+0xd6 k8s.io/component-base@v0.27.4/metrics/registry.go:188 # 0x2324d94 k8s.io/component-base/metrics/prometheus/slis.Register+0x34 k8s.io/component-base@v0.27.4/metrics/prometheus/slis/metrics.go:58 # 0x232539c k8s.io/component-base/metrics/prometheus/slis.SLIMetricsWithReset.Install.func1+0x3c k8s.io/component-base@v0.27.4/metrics/prometheus/slis/routes.go:50 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x232531d sync.(*Once).Do+0x5d sync/once.go:65 # 0x23252de k8s.io/component-base/metrics/prometheus/slis.SLIMetricsWithReset.Install+0x1e k8s.io/component-base@v0.27.4/metrics/prometheus/slis/routes.go:49 # 0x237f5c4 k8s.io/apiserver/pkg/server.installAPI+0x1c4 k8s.io/apiserver@v0.27.4/pkg/server/config.go:986 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 1256 1 @ 0x40c3b8 0x1adeaf0 0x1af47a3 0x1b216e7 0x233be85 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1af47a2 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentMetricHandler+0x2e2 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:267 # 0x1b216e6 k8s.io/component-base/metrics/legacyregistry.HandlerWithReset+0x66 k8s.io/component-base@v0.27.4/metrics/legacyregistry/registry.go:66 # 0x233be84 k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:44 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 1234 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327c72 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327c71 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x311 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:166 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 1062 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308d45 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308d44 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x2c4 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:150 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 964 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308b2e 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308b2d k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0xad k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:143 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 960 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308ae2 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308ae1 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x61 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:142 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 928 1 @ 0x40c3b8 0x1adeaf0 0x1b11497 0x2324db3 0x232539d 0x480462 0x232531e 0x23252df 0x237f5c5 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1b11496 k8s.io/component-base/metrics.(*kubeRegistry).Register+0xd6 k8s.io/component-base@v0.27.4/metrics/registry.go:188 # 0x2324db2 k8s.io/component-base/metrics/prometheus/slis.Register+0x52 k8s.io/component-base@v0.27.4/metrics/prometheus/slis/metrics.go:59 # 0x232539c k8s.io/component-base/metrics/prometheus/slis.SLIMetricsWithReset.Install.func1+0x3c k8s.io/component-base@v0.27.4/metrics/prometheus/slis/routes.go:50 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x232531d sync.(*Once).Do+0x5d sync/once.go:65 # 0x23252de k8s.io/component-base/metrics/prometheus/slis.SLIMetricsWithReset.Install+0x1e k8s.io/component-base@v0.27.4/metrics/prometheus/slis/routes.go:49 # 0x237f5c4 k8s.io/apiserver/pkg/server.installAPI+0x1c4 k8s.io/apiserver@v0.27.4/pkg/server/config.go:986 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 914 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327cbb 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327cba k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x35a k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:167 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 910 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327a0e 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327a0d k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0xad k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:158 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 906 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308b7a 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308b79 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0xf9 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:144 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 894 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327bdd 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327bdc k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x27c k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:164 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 854 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308c0e 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308c0d k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x18d k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:146 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 836 1 @ 0x40c3b8 0x1adeaf0 0x1af562a 0x1af55b0 0x1af4f70 0x1af4865 0x1b216e7 0x233be85 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1af5629 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x209 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1af55af github.com/prometheus/client_golang/prometheus/promhttp.checkLabels+0x18f github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:331 # 0x1af4f6f github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter+0xcf github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/instrument_server.go:141 # 0x1af4864 github.com/prometheus/client_golang/prometheus/promhttp.InstrumentMetricHandler+0x3a4 github.com/prometheus/client_golang@v1.14.0/prometheus/promhttp/http.go:276 # 0x1b216e6 k8s.io/component-base/metrics/legacyregistry.HandlerWithReset+0x66 k8s.io/component-base@v0.27.4/metrics/legacyregistry/registry.go:66 # 0x233be84 k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:44 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 834 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327b45 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327b44 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x1e4 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:162 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 824 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308d8e 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308d8d k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x30d k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:151 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 822 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327a5a 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327a59 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0xf9 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:159 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 818 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327af9 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327af8 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x198 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:161 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 812 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308bc5 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308bc4 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x144 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:145 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 774 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308c5a 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308c59 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x1d9 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:147 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 740 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x23279c2 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x23279c1 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x61 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:157 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 736 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327c29 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327c28 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x2c8 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:165 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 730 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2308ca8 0x480462 0x2308a51 0x2308a35 0x233bf05 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2308ca7 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register.func1+0x227 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:148 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2308a50 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2308a34 k8s.io/apiserver/pkg/storage/etcd3/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/etcd3/metrics/metrics.go:141 # 0x233bf04 k8s.io/apiserver/pkg/server/routes.register+0x24 k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:51 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 730 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327aa8 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327aa7 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x147 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:160 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250 650 1 @ 0x40c3b8 0x1adeaf0 0x1adf925 0x1b1185d 0x2327b91 0x480462 0x2327931 0x2327915 0x233befc 0x233be7e 0x237f57f 0x237e11f 0x23a9e1a 0x23ad805 0x23aba6b 0x5d1c47 0x5d24dd 0x243e61c 0x243e621 0x4413c7 0x475761 # 0x40c3b7 runtime.chanrecv2+0x17 runtime/chan.go:447 # 0x1adeaef github.com/prometheus/client_golang/prometheus.(*Registry).Register+0x24f github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:291 # 0x1adf924 github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister+0x64 github.com/prometheus/client_golang@v1.14.0/prometheus/registry.go:405 # 0x1b1185c k8s.io/component-base/metrics.(*kubeRegistry).MustRegister+0x1fc k8s.io/component-base@v0.27.4/metrics/registry.go:218 # 0x2327b90 k8s.io/apiserver/pkg/storage/cacher/metrics.Register.func1+0x230 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:163 # 0x480461 sync.(*Once).doSlow+0xc1 sync/once.go:74 # 0x2327930 sync.(*Once).Do+0x30 sync/once.go:65 # 0x2327914 k8s.io/apiserver/pkg/storage/cacher/metrics.Register+0x14 k8s.io/apiserver@v0.27.4/pkg/storage/cacher/metrics/metrics.go:156 # 0x233befb k8s.io/apiserver/pkg/server/routes.register+0x1b k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:50 # 0x233be7d k8s.io/apiserver/pkg/server/routes.MetricsWithReset.Install+0x1d k8s.io/apiserver@v0.27.4/pkg/server/routes/metrics.go:43 # 0x237f57e k8s.io/apiserver/pkg/server.installAPI+0x17e k8s.io/apiserver@v0.27.4/pkg/server/config.go:984 # 0x237e11e k8s.io/apiserver/pkg/server.completedConfig.New+0x165e k8s.io/apiserver@v0.27.4/pkg/server/config.go:876 # 0x23a9e19 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerBuilder).Run+0xe79 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/builder.go:284 # 0x23ad804 github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).StartController+0x764 github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:319 # 0x23aba6a github.com/openshift/library-go/pkg/controller/controllercmd.(*ControllerCommandConfig).NewCommandWithContext.func1+0x3ea github.com/openshift/library-go@v0.0.0-20230724150037-c515269de16e/pkg/controller/controllercmd/cmd.go:158 # 0x5d1c46 github.com/spf13/cobra.(*Command).execute+0x846 github.com/spf13/cobra@v1.6.1/command.go:920 # 0x5d24dc github.com/spf13/cobra.(*Command).ExecuteC+0x3bc github.com/spf13/cobra@v1.6.1/command.go:1044 # 0x243e61b github.com/spf13/cobra.(*Command).Execute+0x21b github.com/spf13/cobra@v1.6.1/command.go:968 # 0x243e620 main.main+0x220 github.com/openshift/cluster-etcd-operator/cmd/cluster-etcd-operator/main.go:46 # 0x4413c6 runtime.main+0x206 runtime/proc.go:250