Uploaded image for project: 'A-MQ Messaging-as-a-Service'
  1. A-MQ Messaging-as-a-Service
  2. ENTMQMAAS-2759

Address space controller unable to contact the API server after upgrade

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Major Major
    • None
    • 1.7.0
    • address-controller
    • None
    • False
    • False
    • Workaround Exists
    • Hide

      Pod restart.

      Show
      Pod restart.

      After 1.7.1 respin, it appears that the controller had got into a state where it was unable to contact the API server. Normally the controllers would survive an API server restart without issue, so we suspect there must be a race in the underlying library code.

      2021-09-22T05:44:27.414Z WARN  [Reflector] KubernetesClientException when triggering resync. Pausing for a few seconds before retrying.
      io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53)
      	at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:245)
      	at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:197)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:477)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:435)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:418)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:160)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:665)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:85)
      	at io.enmasse.k8s.api.KubeCrdApi.list(KubeCrdApi.java:46)
      	at io.enmasse.k8s.api.cache.Reflector.resync(Reflector.java:93)
      	at io.enmasse.k8s.api.cache.Reflector.run(Reflector.java:61)
      	at io.enmasse.k8s.api.cache.Controller.lambda$start$0(Controller.java:22)
      	at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: java.net.SocketTimeoutException: timeout
      	at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:656)
      	at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:664)
      	at okhttp3.internal.http2.Http2Stream$FramingSource.read(Http2Stream.java:398)
      	at okhttp3.internal.http2.Http2Codec$StreamFinishingSource.read(Http2Codec.java:205)
      	at okio.RealBufferedSource$1.read(RealBufferedSource.java:439)
      	at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
      	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
      	at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:235)
      	... 12 common frames omitted
      

              Unassigned Unassigned
              rhn-support-fvaleri Federico Valeri
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: