Detected Zookeeper ID 1 Preparing truststore Adding /opt/kafka/cluster-ca-certs/ca.crt to truststore /tmp/zookeeper/cluster.truststore.p12 with alias ca Certificate was added to keystore Preparing truststore is complete Looking for the right CA Found the right CA: /opt/kafka/cluster-ca-certs/ca.crt Preparing keystore for client and quorum listeners Preparing keystore for client and quorum listeners is complete Starting Zookeeper with configuration: # The directory where the snapshot is stored. dataDir=/var/lib/zookeeper/data # Other options 4lw.commands.whitelist=* standaloneEnabled=false reconfigEnabled=true clientPort=12181 clientPortAddress=127.0.0.1 # TLS options serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ssl.clientAuth=need ssl.quorum.clientAuth=need secureClientPort=2181 sslQuorum=true ssl.trustStore.location=/tmp/zookeeper/cluster.truststore.p12 ssl.trustStore.password=KX05ni78Gxcz8-5OKKBrCct_DC3d8w8u ssl.trustStore.type=PKCS12 ssl.quorum.trustStore.location=/tmp/zookeeper/cluster.truststore.p12 ssl.quorum.trustStore.password=KX05ni78Gxcz8-5OKKBrCct_DC3d8w8u ssl.quorum.trustStore.type=PKCS12 ssl.keyStore.location=/tmp/zookeeper/cluster.keystore.p12 ssl.keyStore.password=KX05ni78Gxcz8-5OKKBrCct_DC3d8w8u ssl.keyStore.type=PKCS12 ssl.quorum.keyStore.location=/tmp/zookeeper/cluster.keystore.p12 ssl.quorum.keyStore.password=KX05ni78Gxcz8-5OKKBrCct_DC3d8w8u ssl.quorum.keyStore.type=PKCS12 # Provided configuration tickTime=2000 initLimit=5 syncLimit=2 autopurge.purgeInterval=1 # Zookeeper nodes configuration server.1=openshift-my-cluster-zookeeper-0.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2888:3888:participant;127.0.0.1:12181 server.2=openshift-my-cluster-zookeeper-1.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2888:3888:participant;127.0.0.1:12181 server.3=openshift-my-cluster-zookeeper-2.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2888:3888:participant;127.0.0.1:12181 2020-08-18 02:18:38,304 INFO Reading configuration from: /tmp/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main] 2020-08-18 02:18:38,314 INFO clientPortAddress is 127.0.0.1:12181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main] 2020-08-18 02:18:38,314 INFO secureClientPortAddress is 0.0.0.0:2181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main] 2020-08-18 02:18:38,319 INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util) [main] 2020-08-18 02:18:38,349 INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager) [main] 2020-08-18 02:18:38,350 INFO autopurge.purgeInterval set to 1 (org.apache.zookeeper.server.DatadirCleanupManager) [main] 2020-08-18 02:18:38,353 INFO Purge task started. (org.apache.zookeeper.server.DatadirCleanupManager) [PurgeTask] 2020-08-18 02:18:38,359 INFO Log4j 1.2 jmx support found and enabled. (org.apache.zookeeper.jmx.ManagedUtil) [main] 2020-08-18 02:18:38,367 INFO zookeeper.snapshot.trust.empty : false (org.apache.zookeeper.server.persistence.FileTxnSnapLog) [PurgeTask] 2020-08-18 02:18:38,382 INFO Purge task completed. (org.apache.zookeeper.server.DatadirCleanupManager) [PurgeTask] 2020-08-18 02:18:38,391 INFO Starting quorum peer (org.apache.zookeeper.server.quorum.QuorumPeerMain) [main] 2020-08-18 02:18:38,486 INFO zookeeper.client.portUnification=false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main] 2020-08-18 02:18:38,541 INFO Using org.apache.zookeeper.server.NettyServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [main] 2020-08-18 02:18:38,543 INFO zookeeper.client.portUnification=false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main] 2020-08-18 02:18:38,543 INFO Using org.apache.zookeeper.server.NettyServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [main] 2020-08-18 02:18:38,565 INFO Logging initialized @881ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log) [main] 2020-08-18 02:18:38,659 WARN o.e.j.s.ServletContextHandler@434a63ab{/,null,UNAVAILABLE} contextPath ends with /* (org.eclipse.jetty.server.handler.ContextHandler) [main] 2020-08-18 02:18:38,659 WARN Empty contextPath (org.eclipse.jetty.server.handler.ContextHandler) [main] 2020-08-18 02:18:38,676 INFO zookeeper.snapshot.trust.empty : false (org.apache.zookeeper.server.persistence.FileTxnSnapLog) [main] 2020-08-18 02:18:38,677 INFO Local sessions disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,677 INFO Local session upgrading disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,677 INFO tickTime set to 2000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,677 INFO minSessionTimeout set to 4000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,677 INFO maxSessionTimeout set to 40000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,677 INFO initLimit set to 5 (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,690 INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase) [main] 2020-08-18 02:18:38,693 INFO Using TLS encrypted quorum communication (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,693 INFO Port unification disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,693 INFO QuorumPeer communication is not secured! (SASL auth disabled) (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,693 INFO quorum.cnxn.threads.size set to 20 (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,701 INFO Snapshotting: 0x0 to /var/lib/zookeeper/data/version-2/snapshot.0 (org.apache.zookeeper.server.persistence.FileTxnSnapLog) [main] 2020-08-18 02:18:38,704 INFO currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,706 INFO acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation (org.apache.zookeeper.server.quorum.QuorumPeer) [main] 2020-08-18 02:18:38,707 INFO binding to port /127.0.0.1:12181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main] 2020-08-18 02:18:38,788 INFO bound to port 12181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main] 2020-08-18 02:18:38,789 INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main] 2020-08-18 02:18:38,791 INFO bound to port 2181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main] 2020-08-18 02:18:38,794 INFO jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.8+10-LTS (org.eclipse.jetty.server.Server) [main] 2020-08-18 02:18:38,830 INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session) [main] 2020-08-18 02:18:38,830 INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session) [main] 2020-08-18 02:18:38,832 INFO node0 Scavenging every 600000ms (org.eclipse.jetty.server.session) [main] 2020-08-18 02:18:38,845 INFO Started o.e.j.s.ServletContextHandler@434a63ab{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler) [main] 2020-08-18 02:18:38,855 INFO Started ServerConnector@1e6454ec{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.server.AbstractConnector) [main] 2020-08-18 02:18:38,855 INFO Started @1171ms (org.eclipse.jetty.server.Server) [main] 2020-08-18 02:18:38,855 INFO Started AdminServer on address 0.0.0.0, port 8080 and command URL /commands (org.apache.zookeeper.server.admin.JettyAdminServer) [main] 2020-08-18 02:18:38,863 INFO Election port bind maximum retries is 3 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [main] 2020-08-18 02:18:38,863 INFO Creating TLS-only quorum server socket (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumPeerListener] 2020-08-18 02:18:38,875 INFO 1 is accepting connections now, my election bind port: openshift-my-cluster-zookeeper-0.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.129.2.188:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumPeerListener] 2020-08-18 02:18:38,891 INFO LOOKING (org.apache.zookeeper.server.quorum.QuorumPeer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:38,893 INFO New election. My id = 1, proposed zxid=0x0 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:38,896 INFO Notification: 2 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection) [WorkerReceiver[myid=1]] 2020-08-18 02:18:39,102 INFO Notification time out: 400 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:39,502 INFO Notification time out: 800 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:40,303 INFO Notification time out: 1600 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:40,459 WARN Cannot open channel to 2 at election address openshift-my-cluster-zookeeper-1.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.131.0.137:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=1]-1] java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:289) at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:365) at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:436) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2020-08-18 02:18:40,459 WARN Cannot open channel to 3 at election address openshift-my-cluster-zookeeper-2.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.128.2.191:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=1]-2] java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:289) at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:365) at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:436) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2020-08-18 02:18:41,904 INFO Notification time out: 3200 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:41,905 WARN Cannot open channel to 3 at election address openshift-my-cluster-zookeeper-2.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.128.2.191:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=1]-2] java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:289) at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:365) at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:436) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2020-08-18 02:18:41,905 WARN Cannot open channel to 2 at election address openshift-my-cluster-zookeeper-1.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.131.0.137:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=1]-3] java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:289) at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:365) at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:436) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2020-08-18 02:18:43,363 INFO Received connection request from /10.128.2.191:54888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [openshift-my-cluster-zookeeper-0.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.129.2.188:3888] 2020-08-18 02:18:43,720 INFO Accepted TLS connection from openshift-my-cluster-zookeeper-2.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local/10.128.2.191:54888 - TLSv1.2 - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (org.apache.zookeeper.server.quorum.UnifiedServerSocket) [openshift-my-cluster-zookeeper-0.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.129.2.188:3888] 2020-08-18 02:18:43,732 INFO Notification: 2 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection) [WorkerReceiver[myid=1]] 2020-08-18 02:18:43,733 INFO Notification: 2 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection) [WorkerReceiver[myid=1]] 2020-08-18 02:18:43,934 INFO FOLLOWING (org.apache.zookeeper.server.quorum.QuorumPeer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,938 INFO TCP NoDelay set to: true (org.apache.zookeeper.server.quorum.Learner) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:zookeeper.version=3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:host.name=openshift-my-cluster-zookeeper-0.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:java.version=11.0.8 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:java.vendor=N/A (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:java.home=/usr/lib/jvm/java-11-openjdk-11.0.8.10-0.el7_8.x86_64 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:java.class.path=/opt/kafka/bin/../libs/activation-1.1.1.jar:/opt/kafka/bin/../libs/annotations-13.0.jar:/opt/kafka/bin/../libs/aopalliance-repackaged-2.5.0.jar:/opt/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/bin/../libs/bcpkix-jdk15on-1.62.jar:/opt/kafka/bin/../libs/bcprov-jdk15on-1.60.jar:/opt/kafka/bin/../libs/commons-cli-1.4.jar:/opt/kafka/bin/../libs/commons-lang-2.6.jar:/opt/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/bin/../libs/commons-math3-3.6.1.jar:/opt/kafka/bin/../libs/connect-api-2.6.0.jar:/opt/kafka/bin/../libs/connect-basic-auth-extension-2.6.0.jar:/opt/kafka/bin/../libs/connect-file-2.6.0.jar:/opt/kafka/bin/../libs/connect-json-2.6.0.jar:/opt/kafka/bin/../libs/connect-mirror-2.6.0.jar:/opt/kafka/bin/../libs/connect-mirror-client-2.6.0.jar:/opt/kafka/bin/../libs/connect-runtime-2.6.0.jar:/opt/kafka/bin/../libs/connect-transforms-2.6.0.jar:/opt/kafka/bin/../libs/cruise-control-metrics-reporter-2.5.11.jar:/opt/kafka/bin/../libs/gson-2.8.6.jar:/opt/kafka/bin/../libs/guava-20.0.jar:/opt/kafka/bin/../libs/hamcrest-core-1.3.jar:/opt/kafka/bin/../libs/hk2-api-2.5.0.jar:/opt/kafka/bin/../libs/hk2-locator-2.5.0.jar:/opt/kafka/bin/../libs/hk2-utils-2.5.0.jar:/opt/kafka/bin/../libs/jackson-annotations-2.10.2.jar:/opt/kafka/bin/../libs/jackson-core-2.10.2.jar:/opt/kafka/bin/../libs/jackson-databind-2.10.2.jar:/opt/kafka/bin/../libs/jackson-dataformat-csv-2.10.2.jar:/opt/kafka/bin/../libs/jackson-datatype-jdk8-2.10.2.jar:/opt/kafka/bin/../libs/jackson-jaxrs-base-2.10.2.jar:/opt/kafka/bin/../libs/jackson-jaxrs-json-provider-2.10.2.jar:/opt/kafka/bin/../libs/jackson-module-jaxb-annotations-2.10.2.jar:/opt/kafka/bin/../libs/jackson-module-paranamer-2.10.2.jar:/opt/kafka/bin/../libs/jackson-module-scala_2.12-2.10.2.jar:/opt/kafka/bin/../libs/jaeger-client-1.1.0.jar:/opt/kafka/bin/../libs/jaeger-core-1.1.0.jar:/opt/kafka/bin/../libs/jaeger-thrift-1.1.0.jar:/opt/kafka/bin/../libs/jaeger-tracerresolver-1.1.0.jar:/opt/kafka/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka/bin/../libs/jakarta.annotation-api-1.3.4.jar:/opt/kafka/bin/../libs/jakarta.inject-2.5.0.jar:/opt/kafka/bin/../libs/jakarta.ws.rs-api-2.1.5.jar:/opt/kafka/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka/bin/../libs/javassist-3.22.0-CR2.jar:/opt/kafka/bin/../libs/javassist-3.26.0-GA.jar:/opt/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/bin/../libs/jersey-client-2.28.jar:/opt/kafka/bin/../libs/jersey-common-2.28.jar:/opt/kafka/bin/../libs/jersey-container-servlet-2.28.jar:/opt/kafka/bin/../libs/jersey-container-servlet-core-2.28.jar:/opt/kafka/bin/../libs/jersey-hk2-2.28.jar:/opt/kafka/bin/../libs/jersey-media-jaxb-2.28.jar:/opt/kafka/bin/../libs/jersey-server-2.28.jar:/opt/kafka/bin/../libs/jetty-client-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-continuation-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-http-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-io-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-security-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-server-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-servlet-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-servlets-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jetty-util-9.4.24.v20191120.jar:/opt/kafka/bin/../libs/jmx_prometheus_javaagent-0.12.0.jar:/opt/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/bin/../libs/json-smart-1.1.1.jar:/opt/kafka/bin/../libs/jsonevent-layout-1.7.jar:/opt/kafka/bin/../libs/jsr305-3.0.2.jar:/opt/kafka/bin/../libs/junit-4.13.jar:/opt/kafka/bin/../libs/kafka-agent.jar:/opt/kafka/bin/../libs/kafka-clients-2.6.0.jar:/opt/kafka/bin/../libs/kafka-log4j-appender-2.6.0.jar:/opt/kafka/bin/../libs/kafka-oauth-client-0.5.0.jar:/opt/kafka/bin/../libs/kafka-oauth-common-0.5.0.jar:/opt/kafka/bin/../libs/kafka-oauth-keycloak-authorizer-0.5.0.jar:/opt/kafka/bin/../libs/kafka-oauth-server-0.5.0.jar:/opt/kafka/bin/../libs/kafka-streams-2.6.0.jar:/opt/kafka/bin/../libs/kafka-streams-examples-2.6.0.jar:/opt/kafka/bin/../libs/kafka-streams-scala_2.12-2.6.0.jar:/opt/kafka/bin/../libs/kafka-streams-test-utils-2.6.0.jar:/opt/kafka/bin/../libs/kafka-tools-2.6.0.jar:/opt/kafka/bin/../libs/kafka_2.12-2.6.0-sources.jar:/opt/kafka/bin/../libs/kafka_2.12-2.6.0.jar:/opt/kafka/bin/../libs/keycloak-common-10.0.0.jar:/opt/kafka/bin/../libs/keycloak-core-10.0.0.jar:/opt/kafka/bin/../libs/kotlin-stdlib-1.3.50.jar:/opt/kafka/bin/../libs/kotlin-stdlib-common-1.3.50.jar:/opt/kafka/bin/../libs/libthrift-0.13.0.jar:/opt/kafka/bin/../libs/log4j-1.2.17.jar:/opt/kafka/bin/../libs/lz4-java-1.7.1.jar:/opt/kafka/bin/../libs/maven-artifact-3.6.3.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/mirror-maker-agent.jar:/opt/kafka/bin/../libs/netty-buffer-4.1.50.Final.jar:/opt/kafka/bin/../libs/netty-codec-4.1.50.Final.jar:/opt/kafka/bin/../libs/netty-common-4.1.50.Final.jar:/opt/kafka/bin/../libs/netty-handler-4.1.50.Final.jar:/opt/kafka/bin/../libs/netty-resolver-4.1.50.Final.jar:/opt/kafka/bin/../libs/netty-transport-4.1.50.Final.jar:/opt/kafka/bin/../libs/netty-transport-native-epoll-4.1.50.Final.jar:/opt/kafka/bin/../libs/netty-transport-native-unix-common-4.1.50.Final.jar:/opt/kafka/bin/../libs/okhttp-4.2.2.jar:/opt/kafka/bin/../libs/okio-2.2.2.jar:/opt/kafka/bin/../libs/opa-authorizer-0.4.1.jar:/opt/kafka/bin/../libs/opentracing-api-0.33.0.jar:/opt/kafka/bin/../libs/opentracing-kafka-client-0.1.12.jar:/opt/kafka/bin/../libs/opentracing-noop-0.33.0.jar:/opt/kafka/bin/../libs/opentracing-tracerresolver-0.1.8.jar:/opt/kafka/bin/../libs/opentracing-util-0.33.0.jar:/opt/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka/bin/../libs/paranamer-2.8.jar:/opt/kafka/bin/../libs/plexus-utils-3.2.1.jar:/opt/kafka/bin/../libs/reflections-0.9.12.jar:/opt/kafka/bin/../libs/rocksdbjni-5.18.4.jar:/opt/kafka/bin/../libs/scala-collection-compat_2.12-2.1.6.jar:/opt/kafka/bin/../libs/scala-java8-compat_2.12-0.9.1.jar:/opt/kafka/bin/../libs/scala-library-2.12.11.jar:/opt/kafka/bin/../libs/scala-logging_2.12-3.9.2.jar:/opt/kafka/bin/../libs/scala-reflect-2.12.11.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.30.jar:/opt/kafka/bin/../libs/slf4j-log4j12-1.7.30.jar:/opt/kafka/bin/../libs/snappy-java-1.1.7.3.jar:/opt/kafka/bin/../libs/tracing-agent.jar:/opt/kafka/bin/../libs/validation-api-2.0.1.Final.jar:/opt/kafka/bin/../libs/zookeeper-3.5.8.jar:/opt/kafka/bin/../libs/zookeeper-jute-3.5.8.jar:/opt/kafka/bin/../libs/zstd-jni-1.4.4-7.jar (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:java.io.tmpdir=/tmp (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:java.compiler= (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:os.name=Linux (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:os.version=4.18.0-193.14.3.el8_2.x86_64 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:user.name=1000600000 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:user.home=/ (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:user.dir=/opt/kafka (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:os.memory.free=338MB (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:os.memory.max=3754MB (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,943 INFO Server environment:os.memory.total=361MB (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,945 INFO minSessionTimeout set to 4000 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,945 INFO maxSessionTimeout set to 40000 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,946 INFO Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /var/lib/zookeeper/data/version-2 snapdir /var/lib/zookeeper/data/version-2 (org.apache.zookeeper.server.ZooKeeperServer) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,946 INFO FOLLOWING - LEADER ELECTION TOOK - 11 MS (org.apache.zookeeper.server.quorum.Learner) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:43,949 WARN Unexpected exception, tries=0, remaining init limit=9999, connecting to openshift-my-cluster-zookeeper-2.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.128.2.191:2888 (org.apache.zookeeper.server.quorum.Learner) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:289) at org.apache.zookeeper.server.quorum.Learner.sockConnect(Learner.java:233) at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:262) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:77) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1253) 2020-08-18 02:18:44,168 INFO Received connection request from /10.131.0.137:50852 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [openshift-my-cluster-zookeeper-0.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.129.2.188:3888] 2020-08-18 02:18:44,491 INFO SSL handshake complete with openshift-my-cluster-zookeeper-1.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.131.0.137:3888 - TLSv1.2 - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=1]-3] 2020-08-18 02:18:44,492 INFO Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2) (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=1]-3] 2020-08-18 02:18:44,580 INFO Accepted TLS connection from openshift-my-cluster-zookeeper-1.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local/10.131.0.137:50852 - TLSv1.2 - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (org.apache.zookeeper.server.quorum.UnifiedServerSocket) [openshift-my-cluster-zookeeper-0.openshift-my-cluster-zookeeper-nodes.kafka-cluster-test.svc/10.129.2.188:3888] 2020-08-18 02:18:44,617 INFO Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection) [WorkerReceiver[myid=1]] 2020-08-18 02:18:44,618 INFO Notification: 2 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection) [WorkerReceiver[myid=1]] 2020-08-18 02:18:45,035 INFO Getting a diff from the leader 0x0 (org.apache.zookeeper.server.quorum.Learner) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:45,040 INFO Learner received NEWLEADER message (org.apache.zookeeper.server.quorum.Learner) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:45,071 INFO Learner received UPTODATE message (org.apache.zookeeper.server.quorum.Learner) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:45,080 INFO Configuring CommitProcessor with 1 worker threads. (org.apache.zookeeper.server.quorum.CommitProcessor) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:52,650 INFO The list of known four letter word commands is : [{1936881266=srvr, 1937006964=stat, 2003003491=wchc, 1685417328=dump, 1668445044=crst, 1936880500=srst, 1701738089=envi, 1668247142=conf, -720899=telnet close, 2003003507=wchs, 2003003504=wchp, 1684632179=dirs, 1668247155=cons, 1835955314=mntr, 1769173615=isro, 1920298859=ruok, 1735683435=gtmk, 1937010027=stmk}] (org.apache.zookeeper.server.command.FourLetterCommands) [nioEventLoopGroup-4-1] 2020-08-18 02:18:52,650 INFO The list of enabled four letter word commands is : [[wchs, stat, wchp, dirs, stmk, conf, ruok, mntr, srvr, wchc, envi, srst, isro, dump, gtmk, telnet close, crst, cons]] (org.apache.zookeeper.server.command.FourLetterCommands) [nioEventLoopGroup-4-1] 2020-08-18 02:18:52,650 INFO Processing ruok command from /127.0.0.1:47266 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-1] 2020-08-18 02:18:53,614 INFO Processing ruok command from /127.0.0.1:47282 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-2] 2020-08-18 02:18:57,370 WARN Got zxid 0x100000001 expected 0x1 (org.apache.zookeeper.server.quorum.Learner) [QuorumPeer[myid=1](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)] 2020-08-18 02:18:57,371 INFO Creating new log file: log.100000001 (org.apache.zookeeper.server.persistence.FileTxnLog) [SyncThread:1] 2020-08-18 02:19:02,632 INFO Processing ruok command from /127.0.0.1:47422 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-1] 2020-08-18 02:19:03,627 INFO Processing ruok command from /127.0.0.1:47440 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-2] 2020-08-18 02:19:12,636 INFO Processing ruok command from /127.0.0.1:47580 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-1] 2020-08-18 02:19:13,611 INFO Processing ruok command from /127.0.0.1:47596 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-2] 2020-08-18 02:19:22,618 INFO Processing ruok command from /127.0.0.1:47746 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-1] 2020-08-18 02:19:23,615 INFO Processing ruok command from /127.0.0.1:47762 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-2] 2020-08-18 02:19:30,717 INFO Authenticated Id 'CN=openshift-my-cluster-entity-operator,O=io.strimzi' for Scheme 'x509' (org.apache.zookeeper.server.auth.X509AuthenticationProvider) [nioEventLoopGroup-7-1] 2020-08-18 02:19:30,942 INFO zookeeper.skipACL=="yes", ACL checks will be skipped (org.apache.zookeeper.server.PrepRequestProcessor) [CommitProcWorkThread-1] 2020-08-18 02:19:32,613 INFO Processing ruok command from /127.0.0.1:47902 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-1] 2020-08-18 02:19:33,609 INFO Processing ruok command from /127.0.0.1:47924 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-2] 2020-08-18 02:19:34,731 INFO Authenticated Id 'CN=openshift-my-cluster-entity-operator,O=io.strimzi' for Scheme 'x509' (org.apache.zookeeper.server.auth.X509AuthenticationProvider) [nioEventLoopGroup-7-2] 2020-08-18 02:19:34,794 INFO Authenticated Id 'CN=openshift-my-cluster-entity-operator,O=io.strimzi' for Scheme 'x509' (org.apache.zookeeper.server.auth.X509AuthenticationProvider) [nioEventLoopGroup-7-1] 2020-08-18 02:19:42,616 INFO Processing ruok command from /127.0.0.1:48060 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-1] 2020-08-18 02:19:43,608 INFO Processing ruok command from /127.0.0.1:48074 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-2] 2020-08-18 02:19:52,621 INFO Processing ruok command from /127.0.0.1:48226 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-1] 2020-08-18 02:19:53,605 INFO Processing ruok command from /127.0.0.1:48240 (org.apache.zookeeper.server.NettyServerCnxn) [nioEventLoopGroup-4-2]