-
Bug
-
Resolution: Done
-
Major
-
3.16.0.GA
-
None
-
False
-
None
-
False
-
-
Description of problem:
If check plugin registry pod's logs, there are a lot of exceptions like:
2024-09-23T11:10:05.560Z WARN 47 --- [_WorkerThread-1] i.o.exporter.zipkin.ZipkinSpanExporter : Failed to export spans org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:9411/api/v2/spans": Connect to http://localhost:9411 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused at org.springframework.web.client.RestTemplate.createResourceAccessException(RestTemplate.java:890) ~[spring-web-6.0.21.jar:6.0.21] at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:870) ~[spring-web-6.0.21.jar:6.0.21] at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:765) ~[spring-web-6.0.21.jar:6.0.21] at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:647) ~[spring-web-6.0.21.jar:6.0.21] at org.springframework.boot.actuate.autoconfigure.tracing.zipkin.ZipkinRestTemplateSender$RestTemplateHttpPostCall.doExecute(ZipkinRestTemplateSender.java:68) ~[spring-boot-actuator-autoconfigure-3.1.12.jar:3.1.12] at org.springframework.boot.actuate.autoconfigure.tracing.zipkin.ZipkinRestTemplateSender$RestTemplateHttpPostCall.doEnqueue(ZipkinRestTemplateSender.java:75) ~[spring-boot-actuator-autoconfigure-3.1.12.jar:3.1.12] at zipkin2.Call$Base.enqueue(Call.java:406) ~[zipkin-2.23.2.jar:na] at io.opentelemetry.exporter.zipkin.ZipkinSpanExporter.export(ZipkinSpanExporter.java:79) ~[opentelemetry-exporter-zipkin-1.25.0.jar:1.25.0] at io.micrometer.tracing.otel.bridge.CompositeSpanExporter.lambda$export$3(CompositeSpanExporter.java:88) ~[micrometer-tracing-bridge-otel-1.1.13.jar:1.1.13] at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na] at io.micrometer.tracing.otel.bridge.CompositeSpanExporter.export(CompositeSpanExporter.java:88) ~[micrometer-tracing-bridge-otel-1.1.13.jar:1.1.13] at io.opentelemetry.sdk.trace.export.BatchSpanProcessor$Worker.exportCurrentBatch(BatchSpanProcessor.java:326) ~[opentelemetry-sdk-trace-1.25.0.jar:1.25.0] at io.opentelemetry.sdk.trace.export.BatchSpanProcessor$Worker.run(BatchSpanProcessor.java:244) ~[opentelemetry-sdk-trace-1.25.0.jar:1.25.0] at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na] Caused by: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://localhost:9411 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na] at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[na:na] at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:547) ~[na:na] at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:602) ~[na:na] at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) ~[na:na] at java.base/java.net.Socket.connect(Socket.java:633) ~[na:na] at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.lambda$connectSocket$0(PlainConnectionSocketFactory.java:85) ~[httpclient5-5.2.1.jar:5.2.1] at java.base/java.security.AccessController.doPrivileged(AccessController.java:569) ~[na:na] at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:84) ~[httpclient5-5.2.1.jar:5.2.1] at org.apache.hc.client5.http.socket.ConnectionSocketFactory.connectSocket(ConnectionSocketFactory.java:113) ~[httpclient5-5.2.1.jar:5.2.1] at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:181) ~[httpclient5-5.2.1.jar:5.2.1] at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447) ~[httpclient5-5.2.1.jar:5.2.1] at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162) ~[httpclient5-5.2.1.jar:5.2.1] at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172) ~[httpclient5-5.2.1.jar:5.2.1] at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142) ~[httpclient5-5.2.1.jar:5.2.1]
Errors like this happen after installing an extension in a workspace. It doesn't break the functionality, but it's hard to monitor logs.
We need to identify the cause of these errors and resolve the issue
Prerequisites (if any, like setup, operators/versions):
DS 3.16 with Embedded plugin registry
Steps to Reproduce
# Create a workspace and install a couple extensions
- Check plugin registry pod's logs
Actual results:
Failed to export spans exceptions are in the logs
Expected results:
No exceptions
Reproducibility (Always/Intermittent/Only Once):
Always
