2020-04-14 22:47:22 INFO Main:58 - ClusterOperator 0.18.0-SNAPSHOT is starting 2020-04-14 22:47:23 DEBUG InternalLoggerFactory:45 - Using SLF4J as the default logging framework 2020-04-14 22:47:23 DEBUG ResourceLeakDetector:130 - -Dio.netty.leakDetection.level: simple 2020-04-14 22:47:23 DEBUG ResourceLeakDetector:131 - -Dio.netty.leakDetection.targetRecords: 4 2020-04-14 22:47:23 DEBUG InternalThreadLocalMap:54 - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 2020-04-14 22:47:23 DEBUG InternalThreadLocalMap:57 - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 2020-04-14 22:47:23 DEBUG MultithreadEventLoopGroup:44 - -Dio.netty.eventLoopThreads: 2 2020-04-14 22:47:23 DEBUG NioEventLoop:106 - -Dio.netty.noKeySetOptimization: false 2020-04-14 22:47:23 DEBUG NioEventLoop:107 - -Dio.netty.selectorAutoRebuildThreshold: 512 2020-04-14 22:47:23 DEBUG PlatformDependent0:396 - -Dio.netty.noUnsafe: false 2020-04-14 22:47:23 DEBUG PlatformDependent0:852 - Java version: 8 2020-04-14 22:47:23 DEBUG PlatformDependent0:121 - sun.misc.Unsafe.theUnsafe: available 2020-04-14 22:47:23 DEBUG PlatformDependent0:145 - sun.misc.Unsafe.copyMemory: available 2020-04-14 22:47:23 DEBUG PlatformDependent0:183 - java.nio.Buffer.address: available 2020-04-14 22:47:23 DEBUG PlatformDependent0:244 - direct buffer constructor: available 2020-04-14 22:47:23 DEBUG PlatformDependent0:314 - java.nio.Bits.unaligned: available, true 2020-04-14 22:47:23 DEBUG PlatformDependent0:379 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9 2020-04-14 22:47:23 DEBUG PlatformDependent0:386 - java.nio.DirectByteBuffer.(long, int): available 2020-04-14 22:47:23 DEBUG PlatformDependent:1030 - sun.misc.Unsafe: available 2020-04-14 22:47:23 DEBUG PlatformDependent:1149 - -Dio.netty.tmpdir: /tmp (java.io.tmpdir) 2020-04-14 22:47:23 DEBUG PlatformDependent:1228 - -Dio.netty.bitMode: 64 (sun.arch.data.model) 2020-04-14 22:47:23 DEBUG PlatformDependent:174 - -Dio.netty.maxDirectMemory: 127729664 bytes 2020-04-14 22:47:23 DEBUG PlatformDependent:181 - -Dio.netty.uninitializedArrayAllocationThreshold: -1 2020-04-14 22:47:23 DEBUG CleanerJava6:92 - java.nio.ByteBuffer.cleaner(): available 2020-04-14 22:47:23 DEBUG PlatformDependent:201 - -Dio.netty.noPreferDirect: false 2020-04-14 22:47:23 DEBUG PlatformDependent:891 - org.jctools-core.MpscChunkedArrayQueue: available 2020-04-14 22:47:23 DEBUG DefaultDnsServerAddressStreamProvider:82 - Default DNS servers: [/172.30.0.2:53] (sun.net.dns.ResolverConfiguration) 2020-04-14 22:47:23 DEBUG NetUtil:139 - -Djava.net.preferIPv4Stack: false 2020-04-14 22:47:23 DEBUG NetUtil:140 - -Djava.net.preferIPv6Addresses: false 2020-04-14 22:47:23 DEBUG NetUtil:224 - Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo) 2020-04-14 22:47:23 DEBUG NetUtil:271 - /proc/sys/net/core/somaxconn: 128 2020-04-14 22:47:23 DEBUG Config:465 - Trying to configure client from Kubernetes config... 2020-04-14 22:47:23 DEBUG Config:491 - Did not find Kubernetes config at: [/.kube/config]. Ignoring. 2020-04-14 22:47:23 DEBUG Config:395 - Trying to configure client from service account... 2020-04-14 22:47:23 DEBUG Config:400 - Found service account host and port: 172.30.0.1:443 2020-04-14 22:47:23 DEBUG Config:406 - Found service account ca cert at: [/var/run/secrets/kubernetes.io/serviceaccount/ca.crt]. 2020-04-14 22:47:23 DEBUG Config:414 - Found service account token at: [/var/run/secrets/kubernetes.io/serviceaccount/token]. 2020-04-14 22:47:23 DEBUG Config:605 - Trying to configure client namespace from Kubernetes service account namespace path... 2020-04-14 22:47:23 DEBUG Config:610 - Found service account namespace at: [/var/run/secrets/kubernetes.io/serviceaccount/namespace]. 2020-04-14 22:47:24 DEBUG InternalLoggerFactory:61 - Using SLF4J as the default logging framework 2020-04-14 22:47:24 DEBUG PlatformFeaturesAvailability:141 - https://172.30.0.1/apis/route.openshift.io/v1 returned 200. This API is supported. 2020-04-14 22:47:24 DEBUG PlatformFeaturesAvailability:141 - https://172.30.0.1/apis/build.openshift.io/v1 returned 200. This API is supported. 2020-04-14 22:47:24 DEBUG PlatformFeaturesAvailability:141 - https://172.30.0.1/apis/apps.openshift.io/v1 returned 200. This API is supported. 2020-04-14 22:47:24 DEBUG PlatformFeaturesAvailability:141 - https://172.30.0.1/apis/image.openshift.io/v1 returned 200. This API is supported. 2020-04-14 22:47:24 INFO Main:76 - Environment facts gathered: ClusterOperatorConfig(KubernetesVersion=1.11,OpenShiftRoutes=true,OpenShiftBuilds=true,OpenShiftImageStreams=true,OpenShiftDeploymentConfigs=true) 2020-04-14 22:47:24 INFO Main:210 - Using config: PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin STRIMZI_IMAGE_PULL_POLICY: IfNotPresent STRIMZI_DEFAULT_KAFKA_BRIDGE_IMAGE: strimzi/kafka-bridge:0.15.2 TINI_SHA256: 12d20136605531b09a2c2dac02ccee85e1b874eb322ef6baf7561cd93f93c855 STRIMZI_KAFKA_MIRROR_MAKER_IMAGES: 2.3.1=docker.io/strimzi/kafka:latest-kafka-2.3.1 2.4.0=docker.io/strimzi/kafka:latest-kafka-2.4.0 KUBERNETES_PORT: tcp://172.30.0.1:443 STRIMZI_DEFAULT_TOPIC_OPERATOR_IMAGE: docker.io/strimzi/operator:latest STRIMZI_FULL_RECONCILIATION_INTERVAL_MS: 30000 KUBERNETES_SERVICE_HOST: 172.30.0.1 JAVA_CLASSPATH: lib/io.strimzi.cluster-operator-0.18.0-SNAPSHOT.jar:lib/io.prometheus.simpleclient_common-0.7.0.jar:lib/com.github.luben.zstd-jni-1.4.3-1.jar:lib/io.netty.netty-handler-4.1.45.Final.jar:lib/org.keycloak.keycloak-core-7.0.0.jar:lib/io.netty.netty-codec-http-4.1.45.Final.jar:lib/org.bouncycastle.bcpkix-jdk15on-1.60.jar:lib/org.quartz-scheduler.quartz-2.2.1.jar:lib/io.strimzi.operator-common-0.18.0-SNAPSHOT.jar:lib/org.bouncycastle.bcprov-jdk15on-1.60.jar:lib/com.squareup.okio.okio-1.15.0.jar:lib/io.netty.netty-buffer-4.1.45.Final.jar:lib/org.yaml.snakeyaml-1.24.jar:lib/io.fabric8.openshift-client-4.6.4.jar:lib/io.strimzi.api-0.18.0-SNAPSHOT.jar:lib/io.netty.netty-common-4.1.45.Final.jar:lib/org.apache.logging.log4j.log4j-api-2.13.0.jar:lib/org.xerial.snappy.snappy-java-1.1.7.3.jar:lib/org.hdrhistogram.HdrHistogram-2.1.11.jar:lib/io.prometheus.simpleclient-0.7.0.jar:lib/io.strimzi.kafka-oauth-common-0.4.0.jar:lib/org.apache.yetus.audience-annotations-0.5.0.jar:lib/com.fasterxml.jackson.dataformat.jackson-dataformat-yaml-2.10.2.jar:lib/org.keycloak.keycloak-common-7.0.0.jar:lib/io.strimzi.certificate-manager-0.18.0-SNAPSHOT.jar:lib/io.netty.netty-codec-4.1.45.Final.jar:lib/io.micrometer.micrometer-core-1.3.1.jar:lib/io.strimzi.kafka-oauth-server-0.4.0.jar:lib/jakarta.activation.jakarta.activation-api-1.2.1.jar:lib/io.vertx.vertx-core-3.8.5.jar:lib/io.netty.netty-codec-dns-4.1.45.Final.jar:lib/io.fabric8.kubernetes-model-4.6.4.jar:lib/io.netty.netty-codec-socks-4.1.45.Final.jar:lib/com.github.mifmif.generex-1.0.2.jar:lib/io.netty.netty-resolver-4.1.45.Final.jar:lib/org.apache.zookeeper.zookeeper-jute-3.5.6.jar:lib/io.netty.netty-handler-proxy-4.1.45.Final.jar:lib/com.squareup.okhttp3.logging-interceptor-3.12.6.jar:lib/io.netty.netty-transport-native-epoll-4.1.45.Final-linux-x86_64.jar:lib/io.netty.netty-transport-native-unix-common-4.1.45.Final.jar:lib/org.apache.zookeeper.zookeeper-3.5.6.jar:lib/dk.brics.automaton.automaton-1.11-8.jar:lib/io.vertx.vertx-micrometer-metrics-3.8.5.jar:lib/com.fasterxml.jackson.core.jackson-core-2.10.2.jar:lib/io.netty.netty-transport-4.1.45.Final.jar:lib/io.netty.netty-transport-native-epoll-4.1.45.Final.jar:lib/jakarta.xml.bind.jakarta.xml.bind-api-2.3.2.jar:lib/org.apache.logging.log4j.log4j-slf4j-impl-2.13.0.jar:lib/com.fasterxml.jackson.core.jackson-annotations-2.10.2.jar:lib/io.fabric8.zjsonpatch-0.3.0.jar:lib/org.lz4.lz4-java-1.6.0.jar:lib/io.strimzi.crd-annotations-0.18.0-SNAPSHOT.jar:lib/io.fabric8.kubernetes-client-4.6.4.jar:lib/com.fasterxml.jackson.module.jackson-module-jaxb-annotations-2.10.2.jar:lib/com.squareup.okhttp3.okhttp-3.12.6.jar:lib/io.netty.netty-codec-http2-4.1.45.Final.jar:lib/org.apache.logging.log4j.log4j-core-2.13.0.jar:lib/io.fabric8.kubernetes-model-common-4.6.4.jar:lib/com.fasterxml.jackson.core.jackson-databind-2.10.2.jar:lib/io.netty.netty-resolver-dns-4.1.45.Final.jar:lib/io.strimzi.kafka-oauth-client-0.4.0.jar:lib/org.slf4j.slf4j-api-1.7.25.jar:lib/org.latencyutils.LatencyUtils-2.0.3.jar:lib/io.strimzi.config-model-0.18.0-SNAPSHOT.jar:lib/org.apache.kafka.kafka-clients-2.4.0.jar:lib/io.micrometer.micrometer-registry-prometheus-1.3.1.jar STRIMZI_KAFKA_CONNECT_S2I_IMAGES: 2.3.1=docker.io/strimzi/kafka:latest-kafka-2.3.1 2.4.0=docker.io/strimzi/kafka:latest-kafka-2.4.0 STRIMZI_DEFAULT_TLS_SIDECAR_KAFKA_IMAGE: docker.io/strimzi/kafka:latest-kafka-2.4.0 TINI_VERSION: v0.18.0 STRIMZI_HOME: /opt/strimzi PWD: /opt/strimzi KUBERNETES_PORT_443_TCP: tcp://172.30.0.1:443 STRIMZI_OPERATION_TIMEOUT_MS: 300000 JAVA_MAIN: io.strimzi.operator.cluster.Main KUBERNETES_PORT_443_TCP_ADDR: 172.30.0.1 STRIMZI_VERSION: 0.18.0-SNAPSHOT STRIMZI_DEFAULT_JMXTRANS_IMAGE: docker.io/strimzi/jmxtrans:latest STRIMZI_NAMESPACE: kafka-cluster-test STRIMZI_KAFKA_MIRROR_MAKER_2_IMAGES: 2.4.0=docker.io/strimzi/kafka:latest-kafka-2.4.0 KUBERNETES_PORT_443_TCP_PROTO: tcp STRIMZI_DEFAULT_USER_OPERATOR_IMAGE: docker.io/strimzi/operator:latest KUBERNETES_SERVICE_PORT: 443 STRIMZI_DEFAULT_KAFKA_EXPORTER_IMAGE: docker.io/strimzi/kafka:latest-kafka-2.4.0 STRIMZI_DEFAULT_KAFKA_INIT_IMAGE: docker.io/strimzi/operator:latest STRIMZI_LOG_LEVEL: DEBUG HOSTNAME: strimzi-cluster-operator-689f866d6b-f6rf9 STRIMZI_KAFKA_CONNECT_IMAGES: 2.3.1=docker.io/strimzi/kafka:latest-kafka-2.3.1 2.4.0=docker.io/strimzi/kafka:latest-kafka-2.4.0 STRIMZI_KAFKA_IMAGES: 2.3.1=docker.io/strimzi/kafka:latest-kafka-2.3.1 2.4.0=docker.io/strimzi/kafka:latest-kafka-2.4.0 KUBERNETES_PORT_443_TCP_PORT: 443 STRIMZI_DEFAULT_TLS_SIDECAR_ENTITY_OPERATOR_IMAGE: docker.io/strimzi/kafka:latest-kafka-2.4.0 KUBERNETES_SERVICE_PORT_HTTPS: 443 SHLVL: 0 HOME: / STRIMZI_DEFAULT_TLS_SIDECAR_ZOOKEEPER_IMAGE: docker.io/strimzi/kafka:latest-kafka-2.4.0 MALLOC_ARENA_MAX: 2 2020-04-14 22:47:24 INFO ClusterOperator:84 - Creating ClusterOperator for namespace kafka-cluster-test 2020-04-14 22:47:24 INFO ClusterOperator:101 - Starting ClusterOperator for namespace kafka-cluster-test 2020-04-14 22:47:24 DEBUG WatchConnectionManager:112 - Connecting websocket ... io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@4349c8cb 2020-04-14 22:47:25 DEBUG WatchConnectionManager:167 - WebSocket successfully opened 2020-04-14 22:47:25 INFO ClusterOperator:115 - Opened watch for Kafka operator 2020-04-14 22:47:25 DEBUG WatchConnectionManager:112 - Connecting websocket ... io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@13e5420d 2020-04-14 22:47:25 DEBUG WatchConnectionManager:167 - WebSocket successfully opened 2020-04-14 22:47:25 INFO ClusterOperator:115 - Opened watch for KafkaMirrorMaker operator 2020-04-14 22:47:25 DEBUG WatchConnectionManager:112 - Connecting websocket ... io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@658f8f84 2020-04-14 22:47:25 DEBUG WatchConnectionManager:167 - WebSocket successfully opened 2020-04-14 22:47:25 INFO ClusterOperator:115 - Opened watch for KafkaConnect operator 2020-04-14 22:47:25 DEBUG WatchConnectionManager:112 - Connecting websocket ... io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@1213e698 2020-04-14 22:47:25 DEBUG WatchConnectionManager:167 - WebSocket successfully opened 2020-04-14 22:47:25 INFO ClusterOperator:115 - Opened watch for KafkaBridge operator 2020-04-14 22:47:25 DEBUG WatchConnectionManager:112 - Connecting websocket ... io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@4274a345 2020-04-14 22:47:25 DEBUG WatchConnectionManager:167 - WebSocket successfully opened 2020-04-14 22:47:25 INFO ClusterOperator:115 - Opened watch for KafkaMirrorMaker2 operator 2020-04-14 22:47:25 DEBUG WatchConnectionManager:112 - Connecting websocket ... io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@1d359dfe 2020-04-14 22:47:25 DEBUG WatchConnectionManager:167 - WebSocket successfully opened 2020-04-14 22:47:25 INFO ClusterOperator:115 - Opened watch for KafkaConnectS2I operator 2020-04-14 22:47:25 DEBUG WatchConnectionManager:112 - Connecting websocket ... io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@b448ef5 2020-04-14 22:47:25 DEBUG WatchConnectionManager:167 - WebSocket successfully opened 2020-04-14 22:47:25 INFO ClusterOperator:125 - Setting up periodic reconciliation for namespace kafka-cluster-test 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:142 - -Dio.netty.allocator.numHeapArenas: 1 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:143 - -Dio.netty.allocator.numDirectArenas: 1 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:145 - -Dio.netty.allocator.pageSize: 8192 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:150 - -Dio.netty.allocator.maxOrder: 11 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:154 - -Dio.netty.allocator.chunkSize: 16777216 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:155 - -Dio.netty.allocator.tinyCacheSize: 512 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:156 - -Dio.netty.allocator.smallCacheSize: 256 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:157 - -Dio.netty.allocator.normalCacheSize: 64 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:158 - -Dio.netty.allocator.maxCachedBufferCapacity: 32768 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:159 - -Dio.netty.allocator.cacheTrimInterval: 8192 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:160 - -Dio.netty.allocator.cacheTrimIntervalMillis: 0 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:161 - -Dio.netty.allocator.useCacheForAllThreads: true 2020-04-14 22:47:25 DEBUG PooledByteBufAllocator:162 - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023 2020-04-14 22:47:25 DEBUG DefaultChannelId:79 - -Dio.netty.processId: 10 (auto-detected) 2020-04-14 22:47:25 DEBUG DefaultChannelId:101 - -Dio.netty.machineId: 02:42:ac:ff:fe:11:00:06 (auto-detected) 2020-04-14 22:47:25 DEBUG ByteBufUtil:86 - -Dio.netty.allocator.type: pooled 2020-04-14 22:47:25 DEBUG ByteBufUtil:95 - -Dio.netty.threadLocalDirectBufferSize: 0 2020-04-14 22:47:25 DEBUG ByteBufUtil:98 - -Dio.netty.maxThreadLocalCharBufferSize: 16384 2020-04-14 22:47:25 INFO ClusterOperator:186 - ClusterOperator is now ready (health server listening on 8080) 2020-04-14 22:47:25 INFO Main:146 - Cluster Operator verticle started in namespace kafka-cluster-test 2020-04-14 22:47:33 DEBUG Recycler:97 - -Dio.netty.recycler.maxCapacityPerThread: 4096 2020-04-14 22:47:33 DEBUG Recycler:98 - -Dio.netty.recycler.maxSharedCapacityFactor: 2 2020-04-14 22:47:33 DEBUG Recycler:99 - -Dio.netty.recycler.linkCapacity: 16 2020-04-14 22:47:33 DEBUG Recycler:100 - -Dio.netty.recycler.ratio: 8 2020-04-14 22:47:33 DEBUG AbstractByteBuf:63 - -Dio.netty.buffer.checkAccessible: true 2020-04-14 22:47:33 DEBUG AbstractByteBuf:64 - -Dio.netty.buffer.checkBounds: true 2020-04-14 22:47:33 DEBUG ResourceLeakDetectorFactory:195 - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@2d6d08b9 2020-04-14 22:47:33 DEBUG ZlibCodecFactory:39 - -Dio.netty.noJdkZlibDecoder: false 2020-04-14 22:47:33 DEBUG ZlibCodecFactory:42 - -Dio.netty.noJdkZlibEncoder: false 2020-04-14 22:47:46 INFO OperatorWatcher:40 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was ADDED 2020-04-14 22:47:46 DEBUG AbstractOperator:226 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 22:47:46 WARN VersionUsageUtils:60 - The client is using resource type 'kafkas' with unstable version 'v1beta1' 2020-04-14 22:47:46 INFO AbstractOperator:173 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 22:47:46 DEBUG KafkaAssemblyOperator:504 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Setting the initial status for a new resource 2020-04-14 22:47:46 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/","value":{"conditions":[{"type":"NotReady","status":"True","lastTransitionTime":"2020-04-14T22:47:46+0000","reason":"Creating","message":"Kafka cluster is being deployed"}],"observedGeneration":0}} 2020-04-14 22:47:46 DEBUG StatusDiff:43 - Current Status path / has value 2020-04-14 22:47:46 DEBUG StatusDiff:44 - Desired Status path / has value 2020-04-14 22:47:46 INFO OperatorWatcher:40 - Reconciliation #1(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was MODIFIED 2020-04-14 22:47:46 DEBUG KafkaAssemblyOperator:466 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Completed status update 2020-04-14 22:47:46 DEBUG Ca:646 - cluster-ca: CA key (in my-cluster-cluster-ca) and certificate (in my-cluster-cluster-ca-cert) needs to be created: CA key secret my-cluster-cluster-ca is missing or lacking data.ca\.key 2020-04-14 22:47:46 DEBUG Ca:512 - cluster-ca renewalType CREATE 2020-04-14 22:47:46 DEBUG Ca:886 - Generating CA with subject=/O=io.strimzi/CN=cluster-ca v0 2020-04-14 22:47:46 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -x509, -new, -days, 365, -batch, -nodes, -out, /tmp/tls762159116151381010cluster-ca v0-cert, -keyout, /tmp/tls8309675051776725569cluster-ca v0-key, -subj, /O=io.strimzi/CN=cluster-ca v0] 2020-04-14 22:47:47 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .....................................................................+++ ...............................................+++ unable to write 'random state' writing new private key to '/tmp/tls8309675051776725569cluster-ca v0-key' ----- 2020-04-14 22:47:47 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:47 DEBUG Ca:552 - cluster-ca: CA key (in my-cluster-cluster-ca) and certificate (in my-cluster-cluster-ca-cert) created 2020-04-14 22:47:47 DEBUG Ca:646 - clients-ca: CA key (in my-cluster-clients-ca) and certificate (in my-cluster-clients-ca-cert) needs to be created: CA key secret my-cluster-clients-ca is missing or lacking data.ca\.key 2020-04-14 22:47:47 DEBUG Ca:512 - clients-ca renewalType CREATE 2020-04-14 22:47:47 DEBUG Ca:886 - Generating CA with subject=/O=io.strimzi/CN=clients-ca v0 2020-04-14 22:47:47 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -x509, -new, -days, 365, -batch, -nodes, -out, /tmp/tls1211226730709530438clients-ca v0-cert, -keyout, /tmp/tls3375968625892973239clients-ca v0-key, -subj, /O=io.strimzi/CN=clients-ca v0] 2020-04-14 22:47:47 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .+++ ....................+++ unable to write 'random state' writing new private key to '/tmp/tls3375968625892973239clients-ca v0-key' ----- 2020-04-14 22:47:47 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:47 DEBUG Ca:552 - clients-ca: CA key (in my-cluster-clients-ca) and certificate (in my-cluster-clients-ca-cert) created 2020-04-14 22:47:47 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert does not exist, creating it 2020-04-14 22:47:47 DEBUG SecretOperator:195 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been created 2020-04-14 22:47:47 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-ca does not exist, creating it 2020-04-14 22:47:47 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-clients-ca-cert does not exist, creating it 2020-04-14 22:47:47 DEBUG SecretOperator:195 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been created 2020-04-14 22:47:47 DEBUG SecretOperator:195 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been created 2020-04-14 22:47:47 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-clients-ca does not exist, creating it 2020-04-14 22:47:47 DEBUG SecretOperator:195 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been created 2020-04-14 22:47:47 DEBUG ModelUtils:244 - Certificate for pod cluster-operator need to be regenerated because: certificate doesn't exist yet 2020-04-14 22:47:47 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=cluster-operator with SAN null, signed by CA cluster-ca 2020-04-14 22:47:47 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls9092704551997620086key, -out, /tmp/tls439562965698264168csr, -subj, /O=io.strimzi/CN=cluster-operator] 2020-04-14 22:47:47 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .................+++ ................................................+++ unable to write 'random state' writing new private key to '/tmp/tls9092704551997620086key' ----- 2020-04-14 22:47:47 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:47 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls439562965698264168csr, -CA, /tmp/ca-crt-5120744989206578595.crt, -CAkey, /tmp/ca-key-3532378714056027611.key, -CAcreateserial, -out, /tmp/tls4763045135942929001cert] 2020-04-14 22:47:47 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=cluster-operator Getting CA Private Key unable to write 'random state' 2020-04-14 22:47:47 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:47 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls4763045135942929001cert, -inkey, /tmp/tls9092704551997620086key, -name, cluster-operator, -out, /tmp/tls3434493247908281647p12, -passout, pass:uV07TbDEJZAa] 2020-04-14 22:47:47 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:47:47 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:47 DEBUG ModelUtils:252 - End generating certificates 2020-04-14 22:47:47 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs does not exist, creating it 2020-04-14 22:47:47 DEBUG SecretOperator:195 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been created 2020-04-14 22:47:48 DEBUG NetworkPolicyOperator:99 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper does not exist, creating it 2020-04-14 22:47:48 DEBUG NetworkPolicyOperator:195 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:47:48 DEBUG ServiceAccountOperator:99 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 22:47:48 DEBUG ServiceAccountOperator:195 - ServiceAccount my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:47:48 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-zookeeper-client does not exist, creating it 2020-04-14 22:47:48 DEBUG ServiceOperator:195 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been created 2020-04-14 22:47:48 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-zookeeper-nodes does not exist, creating it 2020-04-14 22:47:48 DEBUG ServiceOperator:195 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been created 2020-04-14 22:47:48 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config does not exist, creating it 2020-04-14 22:47:48 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has been created 2020-04-14 22:47:48 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 22:47:48 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 22:47:48 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 22:47:48 DEBUG Ca:410 - Certificate for my-cluster-zookeeper-0 to generate 2020-04-14 22:47:48 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-zookeeper with SAN {DNS.5=my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local, DNS.6=my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc, DNS.3=my-cluster-zookeeper-client.kafka-cluster-test.svc, DNS.4=my-cluster-zookeeper-client.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-zookeeper-client, DNS.2=my-cluster-zookeeper-client.kafka-cluster-test}, signed by CA cluster-ca 2020-04-14 22:47:48 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls3446763264153195990broker-key, -out, /tmp/tls9034274479565411293broker-csr, -config, /tmp/sna-4263097962495799048.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-zookeeper] 2020-04-14 22:47:48 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ..+++ .+++ unable to write 'random state' writing new private key to '/tmp/tls3446763264153195990broker-key' ----- 2020-04-14 22:47:48 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:48 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls9034274479565411293broker-csr, -CA, /tmp/ca-crt-2796783267587783107.crt, -CAkey, /tmp/ca-key-8576657342241876393.key, -CAcreateserial, -out, /tmp/tls395845905706670020broker-cert, -extfile, /tmp/sna-711742656484693115.conf, -extensions, v3_req] 2020-04-14 22:47:48 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-zookeeper Getting CA Private Key unable to write 'random state' 2020-04-14 22:47:48 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:48 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls395845905706670020broker-cert, -inkey, /tmp/tls3446763264153195990broker-key, -name, my-cluster-zookeeper, -out, /tmp/tls6598786823213218657broker-p12, -passout, pass:IQrqA5L1EVqv] 2020-04-14 22:47:48 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:47:48 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:48 DEBUG Ca:410 - Certificate for my-cluster-zookeeper-1 to generate 2020-04-14 22:47:48 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-zookeeper with SAN {DNS.5=my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local, DNS.6=my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc, DNS.3=my-cluster-zookeeper-client.kafka-cluster-test.svc, DNS.4=my-cluster-zookeeper-client.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-zookeeper-client, DNS.2=my-cluster-zookeeper-client.kafka-cluster-test}, signed by CA cluster-ca 2020-04-14 22:47:48 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls3446763264153195990broker-key, -out, /tmp/tls9034274479565411293broker-csr, -config, /tmp/sna-860127605735386679.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-zookeeper] 2020-04-14 22:47:48 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ..................+++ ........+++ unable to write 'random state' writing new private key to '/tmp/tls3446763264153195990broker-key' ----- 2020-04-14 22:47:48 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:48 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls9034274479565411293broker-csr, -CA, /tmp/ca-crt-1384722482482007007.crt, -CAkey, /tmp/ca-key-7762645240633948833.key, -CAcreateserial, -out, /tmp/tls395845905706670020broker-cert, -extfile, /tmp/sna-3697835440526133778.conf, -extensions, v3_req] 2020-04-14 22:47:48 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-zookeeper Getting CA Private Key unable to write 'random state' 2020-04-14 22:47:48 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:48 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls395845905706670020broker-cert, -inkey, /tmp/tls3446763264153195990broker-key, -name, my-cluster-zookeeper, -out, /tmp/tls6598786823213218657broker-p12, -passout, pass:sj7LHaoHzPIX] 2020-04-14 22:47:48 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:47:48 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:48 DEBUG Ca:410 - Certificate for my-cluster-zookeeper-2 to generate 2020-04-14 22:47:48 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-zookeeper with SAN {DNS.5=my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local, DNS.6=my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc, DNS.3=my-cluster-zookeeper-client.kafka-cluster-test.svc, DNS.4=my-cluster-zookeeper-client.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-zookeeper-client, DNS.2=my-cluster-zookeeper-client.kafka-cluster-test}, signed by CA cluster-ca 2020-04-14 22:47:48 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls3446763264153195990broker-key, -out, /tmp/tls9034274479565411293broker-csr, -config, /tmp/sna-797381595234354376.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-zookeeper] 2020-04-14 22:47:49 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .......................................................................................................+++ .................................................................+++ unable to write 'random state' writing new private key to '/tmp/tls3446763264153195990broker-key' ----- 2020-04-14 22:47:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:49 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls9034274479565411293broker-csr, -CA, /tmp/ca-crt-4026966348698185600.crt, -CAkey, /tmp/ca-key-5837368322432919906.key, -CAcreateserial, -out, /tmp/tls395845905706670020broker-cert, -extfile, /tmp/sna-2941317508452414085.conf, -extensions, v3_req] 2020-04-14 22:47:49 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-zookeeper Getting CA Private Key unable to write 'random state' 2020-04-14 22:47:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:49 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls395845905706670020broker-cert, -inkey, /tmp/tls3446763264153195990broker-key, -name, my-cluster-zookeeper, -out, /tmp/tls6598786823213218657broker-p12, -passout, pass:eZAMNpjI3JPe] 2020-04-14 22:47:49 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:47:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:47:49 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 22:47:49 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes does not exist, creating it 2020-04-14 22:47:49 DEBUG SecretOperator:195 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been created 2020-04-14 22:47:49 WARN VersionUsageUtils:60 - The client is using resource type 'poddisruptionbudgets' with unstable version 'v1beta1' 2020-04-14 22:47:49 DEBUG PodDisruptionBudgetOperator:99 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 22:47:49 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:47:49 DEBUG ZookeeperSetOperator:99 - StatefulSet kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 22:47:49 DEBUG ZookeeperSetOperator:195 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:47:49 DEBUG Util:62 - Waiting for StatefulSet resource my-cluster-zookeeper in namespace kafka-cluster-test to get ready 2020-04-14 22:47:49 DEBUG Util:84 - StatefulSet resource my-cluster-zookeeper in namespace kafka-cluster-test is ready 2020-04-14 22:47:49 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:47:49 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:47:49 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:48:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:49:05 WARN AbstractOperator:246 - Reconciliation #4(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:49:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:49:35 WARN AbstractOperator:246 - Reconciliation #5(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:49:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:50:05 WARN AbstractOperator:246 - Reconciliation #6(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:50:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:50:35 WARN AbstractOperator:246 - Reconciliation #7(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:50:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:51:05 WARN AbstractOperator:246 - Reconciliation #8(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:51:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:51:35 WARN AbstractOperator:246 - Reconciliation #9(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:51:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:52:05 WARN AbstractOperator:246 - Reconciliation #10(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:52:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:52:35 WARN AbstractOperator:246 - Reconciliation #11(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:52:51 DEBUG Util:84 - Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test is ready 2020-04-14 22:52:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:53:05 WARN AbstractOperator:246 - Reconciliation #12(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:53:17 ERROR Util:90 - Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to be ready 2020-04-14 22:53:17 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2020-04-14T22:53:17+0000"} 2020-04-14 22:53:17 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/reason","value":"TimeoutException"} 2020-04-14 22:53:17 DEBUG StatusDiff:43 - Current Status path /conditions/0/reason has value "Creating" 2020-04-14 22:53:17 DEBUG StatusDiff:44 - Desired Status path /conditions/0/reason has value "TimeoutException" 2020-04-14 22:53:17 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/message","value":"Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to be ready"} 2020-04-14 22:53:17 DEBUG StatusDiff:43 - Current Status path /conditions/0/message has value "Kafka cluster is being deployed" 2020-04-14 22:53:17 DEBUG StatusDiff:44 - Desired Status path /conditions/0/message has value "Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to be ready" 2020-04-14 22:53:17 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/observedGeneration","value":1} 2020-04-14 22:53:17 DEBUG StatusDiff:43 - Current Status path /observedGeneration has value 0 2020-04-14 22:53:17 DEBUG StatusDiff:44 - Desired Status path /observedGeneration has value 1 2020-04-14 22:53:17 DEBUG StatusDiff:42 - Status differs: {"op":"add","path":"/listeners","value":[{"type":"plain","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9092}]}]} 2020-04-14 22:53:17 DEBUG StatusDiff:43 - Current Status path /listeners has value 2020-04-14 22:53:17 DEBUG StatusDiff:44 - Desired Status path /listeners has value [{"type":"plain","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9092}]}] 2020-04-14 22:53:17 DEBUG KafkaAssemblyOperator:466 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Completed status update 2020-04-14 22:53:17 DEBUG KafkaAssemblyOperator:229 - Status for my-cluster is up to date 2020-04-14 22:53:17 ERROR AbstractOperator:175 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): createOrUpdate failed io.strimzi.operator.common.operator.resource.TimeoutException: Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to be ready at io.strimzi.operator.common.Util$1.lambda$handle$1(Util.java:91) ~[io.strimzi.operator-common-0.18.0-SNAPSHOT.jar:0.18.0-SNAPSHOT] at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:330) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [io.netty.netty-transport-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242] 2020-04-14 22:53:17 WARN AbstractOperator:329 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Failed to reconcile io.strimzi.operator.common.operator.resource.TimeoutException: Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to be ready at io.strimzi.operator.common.Util$1.lambda$handle$1(Util.java:91) ~[io.strimzi.operator-common-0.18.0-SNAPSHOT.jar:0.18.0-SNAPSHOT] at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:330) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [io.netty.netty-transport-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242] 2020-04-14 22:53:17 DEBUG AbstractOperator:237 - Reconciliation #0(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 22:53:17 INFO OperatorWatcher:40 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was MODIFIED 2020-04-14 22:53:17 DEBUG AbstractOperator:226 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 22:53:17 INFO AbstractOperator:173 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 22:53:17 DEBUG KafkaAssemblyOperator:520 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Status is already set. No need to set initial status 2020-04-14 22:53:17 DEBUG Ca:653 - cluster-ca: The CA certificate in secret my-cluster-cluster-ca-cert already exists and does not need renewing 2020-04-14 22:53:17 DEBUG Ca:512 - cluster-ca renewalType NOOP 2020-04-14 22:53:17 DEBUG Ca:653 - clients-ca: The CA certificate in secret my-cluster-clients-ca-cert already exists and does not need renewing 2020-04-14 22:53:17 DEBUG Ca:512 - clients-ca renewalType NOOP 2020-04-14 22:53:17 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert already exists, patching it 2020-04-14 22:53:17 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca-cert already exists, patching it 2020-04-14 22:53:17 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 22:53:17 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 22:53:17 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca already exists, patching it 2020-04-14 22:53:17 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca already exists, patching it 2020-04-14 22:53:17 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been patched 2020-04-14 22:53:17 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been patched 2020-04-14 22:53:17 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs already exists, patching it 2020-04-14 22:53:17 DEBUG SecretOperator:168 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 22:53:17 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-zookeeper 2020-04-14 22:53:17 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper already exists, patching it 2020-04-14 22:53:18 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:783 - Kafka.spec.kafka.version is unchanged therefore no change to Zookeeper is required 2020-04-14 22:53:18 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:53:18 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-client already exists, patching it 2020-04-14 22:53:18 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been patched 2020-04-14 22:53:18 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 22:53:18 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 22:53:18 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config already exists, patching it 2020-04-14 22:53:18 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:53:18 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 22:53:18 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 22:53:18 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 22:53:18 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-0 already exists 2020-04-14 22:53:18 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-1 already exists 2020-04-14 22:53:18 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-2 already exists 2020-04-14 22:53:18 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 22:53:18 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 22:53:18 DEBUG SecretOperator:168 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 22:53:18 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:53:18 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been deleted 2020-04-14 22:53:18 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:53:18 DEBUG ZookeeperSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/resources"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/1/configMap/defaultMode"} 2020-04-14 22:53:18 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/status"} 2020-04-14 22:53:18 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-zookeeper 2020-04-14 22:53:18 DEBUG ZookeeperSetOperator:168 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 22:53:18 DEBUG ZookeeperSetOperator:74 - Considering rolling update of kafka-cluster-test/my-cluster-zookeeper 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-0. 2020-04-14 22:53:18 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-1. 2020-04-14 22:53:18 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-2. 2020-04-14 22:53:18 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:18 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 22:53:18 DEBUG Util:84 - Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test is ready 2020-04-14 22:53:18 DEBUG Util:84 - Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test is ready 2020-04-14 22:53:18 DEBUG KafkaAssemblyOperator:1538 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Verifying that Zookeeper is configured to run with 3 replicas 2020-04-14 22:53:18 DEBUG ZookeeperScaler:62 - Creating Zookeeper Scaler for cluster my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:53:18 DEBUG Util:62 - Waiting for ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 to get ready 2020-04-14 22:53:18 DEBUG SslHandler:1743 - [id: 0xb7dacc3d, L:/172.17.0.6:43194 - R:my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc/172.17.0.9:2181] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 2020-04-14 22:53:18 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:SyncConnected type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:53:19 DEBUG Util:84 - ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 is ready 2020-04-14 22:53:19 DEBUG ZookeeperScaler:198 - Current Zookeeper configuration is {server.2=127.0.0.1:28881:38881:participant;127.0.0.1:21811, server.1=127.0.0.1:28880:38880:participant;127.0.0.1:21810, server.3=127.0.0.1:28882:38882:participant;127.0.0.1:21812} 2020-04-14 22:53:19 DEBUG ZookeeperScaler:181 - The Zookeeper server configuration is already up to date 2020-04-14 22:53:19 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:Closed type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:53:19 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test to get ready 2020-04-14 22:53:19 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test is ready 2020-04-14 22:53:19 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test to get ready 2020-04-14 22:53:19 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test is ready 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-kafka 2020-04-14 22:53:19 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-kafka already exists, patching it 2020-04-14 22:53:19 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-kafka in namespace kafka-cluster-test has been patched 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:920 - Kafka.spec.kafka.version unchanged 2020-04-14 22:53:19 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 22:53:19 DEBUG ClusterRoleBindingOperator:110 - ClusterRoleBinding strimzi-kafka-cluster-test-my-cluster-kafka-init does not exist, noop 2020-04-14 22:53:19 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-bootstrap already exists, patching it 2020-04-14 22:53:19 DEBUG ServiceOperator:168 - Service my-cluster-kafka-bootstrap in namespace kafka-cluster-test has been patched 2020-04-14 22:53:19 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 22:53:19 DEBUG ServiceOperator:168 - Service my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 22:53:19 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-external-bootstrap does not exist, noop 2020-04-14 22:53:19 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 22:53:19 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 22:53:19 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 22:53:19 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, noop 2020-04-14 22:53:19 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 22:53:19 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 22:53:19 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 22:53:19 DEBUG AbstractModel:739 - Generating certificates 2020-04-14 22:53:19 DEBUG Ca:193 - cluster-ca: Reconciling kafka broker certificates 2020-04-14 22:53:19 DEBUG Ca:361 - Certificate for my-cluster-kafka-0 already exists 2020-04-14 22:53:19 DEBUG Ca:361 - Certificate for my-cluster-kafka-1 already exists 2020-04-14 22:53:19 DEBUG Ca:361 - Certificate for my-cluster-kafka-2 already exists 2020-04-14 22:53:19 DEBUG AbstractModel:747 - End generating certificates 2020-04-14 22:53:19 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-kafka-config already exists, patching it 2020-04-14 22:53:19 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-kafka-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:53:19 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 22:53:19 DEBUG SecretOperator:168 - Secret my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 22:53:19 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-jmx does not exist, noop 2020-04-14 22:53:19 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 22:53:19 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been deleted 2020-04-14 22:53:19 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 22:53:19 DEBUG KafkaSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/resources"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/4/configMap/defaultMode"} 2020-04-14 22:53:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/status"} 2020-04-14 22:53:19 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-kafka 2020-04-14 22:53:19 DEBUG KafkaSetOperator:168 - StatefulSet my-cluster-kafka in namespace kafka-cluster-test has been patched 2020-04-14 22:53:19 DEBUG KafkaRoller:150 - Initial order for rolling restart [0, 1, 2] 2020-04-14 22:53:19 DEBUG KafkaRoller:194 - Considering restart of pod 0 after delay of 0 MILLISECONDS 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:53:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:53:19 DEBUG KafkaRoller:281 - Pod 0 does not need to be restarted 2020-04-14 22:53:19 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 0 to become ready 2020-04-14 22:53:19 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:21 DEBUG PoolThreadCache:250 - Freed 16 thread-local buffer(s) from thread: epollEventLoopGroup-3-1 2020-04-14 22:53:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:53:35 WARN AbstractOperator:246 - Reconciliation #14(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:53:45 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG KafkaRoller:284 - Pod 0 is now ready 2020-04-14 22:53:45 DEBUG KafkaRoller:194 - Considering restart of pod 1 after delay of 0 MILLISECONDS 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:53:45 DEBUG KafkaRoller:281 - Pod 1 does not need to be restarted 2020-04-14 22:53:45 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 1 to become ready 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:45 DEBUG Util:84 - Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG KafkaRoller:284 - Pod 1 is now ready 2020-04-14 22:53:45 DEBUG KafkaRoller:194 - Considering restart of pod 2 after delay of 0 MILLISECONDS 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:53:45 DEBUG KafkaRoller:281 - Pod 2 does not need to be restarted 2020-04-14 22:53:45 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 2 to become ready 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:45 DEBUG Util:84 - Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG KafkaRoller:284 - Pod 2 is now ready 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-0. 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-1. 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:45 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-2. 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:53:45 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG Util:84 - Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG Util:84 - Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test to get ready 2020-04-14 22:53:45 DEBUG Util:84 - Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test to get ready 2020-04-14 22:53:45 DEBUG Util:84 - Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test is ready 2020-04-14 22:53:45 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 22:53:45 DEBUG RoleBindingOperator:111 - RoleBinding kafka-cluster-test/strimzi-my-cluster-topic-operator does not exist, noop 2020-04-14 22:53:45 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-topic-operator-config does not exist, noop 2020-04-14 22:53:45 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-topic-operator-certs does not exist, noop 2020-04-14 22:53:45 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 22:53:45 DEBUG ServiceAccountOperator:99 - ServiceAccount kafka-cluster-test/my-cluster-entity-operator does not exist, creating it 2020-04-14 22:53:45 DEBUG ServiceAccountOperator:195 - ServiceAccount my-cluster-entity-operator in namespace kafka-cluster-test has been created 2020-04-14 22:53:45 DEBUG RoleBindingOperator:99 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-topic-operator does not exist, creating it 2020-04-14 22:53:45 DEBUG RoleBindingOperator:195 - RoleBinding strimzi-my-cluster-entity-topic-operator in namespace kafka-cluster-test has been created 2020-04-14 22:53:45 DEBUG RoleBindingOperator:99 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-user-operator does not exist, creating it 2020-04-14 22:53:45 DEBUG RoleBindingOperator:195 - RoleBinding strimzi-my-cluster-entity-user-operator in namespace kafka-cluster-test has been created 2020-04-14 22:53:45 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-entity-topic-operator-config does not exist, creating it 2020-04-14 22:53:45 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-entity-topic-operator-config in namespace kafka-cluster-test has been created 2020-04-14 22:53:45 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-entity-user-operator-config does not exist, creating it 2020-04-14 22:53:45 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-entity-user-operator-config in namespace kafka-cluster-test has been created 2020-04-14 22:53:45 DEBUG ModelUtils:244 - Certificate for pod entity-operator need to be regenerated because: certificate doesn't exist yet 2020-04-14 22:53:45 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-entity-operator with SAN null, signed by CA cluster-ca 2020-04-14 22:53:45 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls4186877595634222896key, -out, /tmp/tls738443535658494836csr, -subj, /O=io.strimzi/CN=my-cluster-entity-operator] 2020-04-14 22:53:45 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .....................................+++ ..........+++ unable to write 'random state' writing new private key to '/tmp/tls4186877595634222896key' ----- 2020-04-14 22:53:45 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:53:45 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls738443535658494836csr, -CA, /tmp/ca-crt-9147083432003895466.crt, -CAkey, /tmp/ca-key-3737109349225817163.key, -CAcreateserial, -out, /tmp/tls6644064171366711771cert] 2020-04-14 22:53:45 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-entity-operator Getting CA Private Key unable to write 'random state' 2020-04-14 22:53:45 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:53:45 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls6644064171366711771cert, -inkey, /tmp/tls4186877595634222896key, -name, my-cluster-entity-operator, -out, /tmp/tls8313450527775355550p12, -passout, pass:XjULmvJZJazI] 2020-04-14 22:53:45 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:53:45 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:53:45 DEBUG ModelUtils:252 - End generating certificates 2020-04-14 22:53:45 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-entity-operator-certs does not exist, creating it 2020-04-14 22:53:45 DEBUG SecretOperator:195 - Secret my-cluster-entity-operator-certs in namespace kafka-cluster-test has been created 2020-04-14 22:53:45 DEBUG DeploymentOperator:99 - Deployment kafka-cluster-test/my-cluster-entity-operator does not exist, creating it 2020-04-14 22:53:45 DEBUG DeploymentOperator:195 - Deployment my-cluster-entity-operator in namespace kafka-cluster-test has been created 2020-04-14 22:53:45 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 22:53:46 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 22:53:46 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 22:53:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:54:03 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 22:54:03 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 22:54:03 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-exporter-certs does not exist, noop 2020-04-14 22:54:03 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 22:54:03 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 22:54:03 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 22:54:03 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-jmxtrans-config does not exist, noop 2020-04-14 22:54:03 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 22:54:03 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/type","value":"Ready"} 2020-04-14 22:54:03 DEBUG StatusDiff:43 - Current Status path /conditions/0/type has value "NotReady" 2020-04-14 22:54:03 DEBUG StatusDiff:44 - Desired Status path /conditions/0/type has value "Ready" 2020-04-14 22:54:03 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2020-04-14T22:54:03+0000"} 2020-04-14 22:54:03 DEBUG StatusDiff:42 - Status differs: {"op":"remove","path":"/conditions/0/reason"} 2020-04-14 22:54:03 DEBUG StatusDiff:43 - Current Status path /conditions/0/reason has value "TimeoutException" 2020-04-14 22:54:03 DEBUG StatusDiff:44 - Desired Status path /conditions/0/reason has value 2020-04-14 22:54:03 DEBUG StatusDiff:42 - Status differs: {"op":"remove","path":"/conditions/0/message"} 2020-04-14 22:54:03 DEBUG StatusDiff:43 - Current Status path /conditions/0/message has value "Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to be ready" 2020-04-14 22:54:03 DEBUG StatusDiff:44 - Desired Status path /conditions/0/message has value 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:466 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Completed status update 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:229 - Status for my-cluster is up to date 2020-04-14 22:54:03 INFO OperatorWatcher:40 - Reconciliation #16(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was MODIFIED 2020-04-14 22:54:03 INFO AbstractOperator:317 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): reconciled 2020-04-14 22:54:03 DEBUG AbstractOperator:237 - Reconciliation #13(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 22:54:03 DEBUG AbstractOperator:226 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 22:54:03 INFO AbstractOperator:173 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:520 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Status is already set. No need to set initial status 2020-04-14 22:54:03 DEBUG Ca:653 - cluster-ca: The CA certificate in secret my-cluster-cluster-ca-cert already exists and does not need renewing 2020-04-14 22:54:03 DEBUG Ca:512 - cluster-ca renewalType NOOP 2020-04-14 22:54:03 DEBUG Ca:653 - clients-ca: The CA certificate in secret my-cluster-clients-ca-cert already exists and does not need renewing 2020-04-14 22:54:03 DEBUG Ca:512 - clients-ca renewalType NOOP 2020-04-14 22:54:03 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert already exists, patching it 2020-04-14 22:54:03 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca-cert already exists, patching it 2020-04-14 22:54:03 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca already exists, patching it 2020-04-14 22:54:03 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca already exists, patching it 2020-04-14 22:54:03 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs already exists, patching it 2020-04-14 22:54:03 DEBUG SecretOperator:168 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-zookeeper 2020-04-14 22:54:03 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper already exists, patching it 2020-04-14 22:54:03 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 22:54:03 DEBUG KafkaAssemblyOperator:783 - Kafka.spec.kafka.version is unchanged therefore no change to Zookeeper is required 2020-04-14 22:54:03 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:54:03 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-client already exists, patching it 2020-04-14 22:54:03 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 22:54:03 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config already exists, patching it 2020-04-14 22:54:03 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:54:03 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 22:54:03 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 22:54:03 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 22:54:03 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-0 already exists 2020-04-14 22:54:03 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-1 already exists 2020-04-14 22:54:03 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-2 already exists 2020-04-14 22:54:03 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 22:54:03 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 22:54:03 DEBUG SecretOperator:168 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 22:54:03 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:54:03 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been deleted 2020-04-14 22:54:03 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:54:03 DEBUG ZookeeperSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/resources"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/1/configMap/defaultMode"} 2020-04-14 22:54:03 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/status"} 2020-04-14 22:54:03 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-zookeeper 2020-04-14 22:54:04 DEBUG ZookeeperSetOperator:168 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 22:54:04 DEBUG ZookeeperSetOperator:74 - Considering rolling update of kafka-cluster-test/my-cluster-zookeeper 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-0. 2020-04-14 22:54:04 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-1. 2020-04-14 22:54:04 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-2. 2020-04-14 22:54:04 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:04 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 22:54:04 DEBUG Util:84 - Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test is ready 2020-04-14 22:54:04 DEBUG Util:84 - Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test is ready 2020-04-14 22:54:04 DEBUG KafkaAssemblyOperator:1538 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Verifying that Zookeeper is configured to run with 3 replicas 2020-04-14 22:54:04 DEBUG ZookeeperScaler:62 - Creating Zookeeper Scaler for cluster my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:54:04 DEBUG Util:62 - Waiting for ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 to get ready 2020-04-14 22:54:04 DEBUG SslHandler:1743 - [id: 0xe677dc7b, L:/172.17.0.6:38172 - R:my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc/172.17.0.14:2181] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 2020-04-14 22:54:04 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:SyncConnected type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:54:05 DEBUG Util:84 - ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 is ready 2020-04-14 22:54:05 DEBUG ZookeeperScaler:198 - Current Zookeeper configuration is {server.2=127.0.0.1:28881:38881:participant;127.0.0.1:21811, server.1=127.0.0.1:28880:38880:participant;127.0.0.1:21810, server.3=127.0.0.1:28882:38882:participant;127.0.0.1:21812} 2020-04-14 22:54:05 DEBUG ZookeeperScaler:181 - The Zookeeper server configuration is already up to date 2020-04-14 22:54:05 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:Closed type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:54:05 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test to get ready 2020-04-14 22:54:05 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test is ready 2020-04-14 22:54:05 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test to get ready 2020-04-14 22:54:05 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test is ready 2020-04-14 22:54:05 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #15(timer) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-kafka 2020-04-14 22:54:05 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-kafka already exists, patching it 2020-04-14 22:54:05 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-kafka in namespace kafka-cluster-test has been patched 2020-04-14 22:54:05 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:05 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:05 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:05 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 22:54:05 DEBUG KafkaAssemblyOperator:920 - Kafka.spec.kafka.version unchanged 2020-04-14 22:54:05 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 22:54:05 DEBUG ClusterRoleBindingOperator:110 - ClusterRoleBinding strimzi-kafka-cluster-test-my-cluster-kafka-init does not exist, noop 2020-04-14 22:54:05 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-bootstrap already exists, patching it 2020-04-14 22:54:05 DEBUG ServiceOperator:168 - Service my-cluster-kafka-bootstrap in namespace kafka-cluster-test has been patched 2020-04-14 22:54:05 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 22:54:05 DEBUG ServiceOperator:168 - Service my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 22:54:05 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-external-bootstrap does not exist, noop 2020-04-14 22:54:05 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 22:54:05 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 22:54:05 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 22:54:05 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, noop 2020-04-14 22:54:05 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 22:54:05 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 22:54:05 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 22:54:05 DEBUG AbstractModel:739 - Generating certificates 2020-04-14 22:54:05 DEBUG Ca:193 - cluster-ca: Reconciling kafka broker certificates 2020-04-14 22:54:05 DEBUG Ca:361 - Certificate for my-cluster-kafka-0 already exists 2020-04-14 22:54:05 DEBUG Ca:361 - Certificate for my-cluster-kafka-1 already exists 2020-04-14 22:54:05 DEBUG Ca:361 - Certificate for my-cluster-kafka-2 already exists 2020-04-14 22:54:05 DEBUG AbstractModel:747 - End generating certificates 2020-04-14 22:54:05 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-kafka-config already exists, patching it 2020-04-14 22:54:05 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-kafka-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:54:05 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 22:54:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:54:55 DEBUG AbstractOperator:226 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 22:54:55 INFO AbstractOperator:173 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:520 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Status is already set. No need to set initial status 2020-04-14 22:54:55 DEBUG Ca:653 - cluster-ca: The CA certificate in secret my-cluster-cluster-ca-cert already exists and does not need renewing 2020-04-14 22:54:55 DEBUG Ca:512 - cluster-ca renewalType NOOP 2020-04-14 22:54:55 DEBUG Ca:653 - clients-ca: The CA certificate in secret my-cluster-clients-ca-cert already exists and does not need renewing 2020-04-14 22:54:55 DEBUG Ca:512 - clients-ca renewalType NOOP 2020-04-14 22:54:55 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert already exists, patching it 2020-04-14 22:54:55 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca-cert already exists, patching it 2020-04-14 22:54:55 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca already exists, patching it 2020-04-14 22:54:55 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca already exists, patching it 2020-04-14 22:54:55 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs already exists, patching it 2020-04-14 22:54:55 DEBUG SecretOperator:168 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-zookeeper 2020-04-14 22:54:55 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper already exists, patching it 2020-04-14 22:54:55 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:783 - Kafka.spec.kafka.version is unchanged therefore no change to Zookeeper is required 2020-04-14 22:54:55 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:54:55 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-client already exists, patching it 2020-04-14 22:54:55 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 22:54:55 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config already exists, patching it 2020-04-14 22:54:55 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:54:55 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 22:54:55 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 22:54:55 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 22:54:55 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-0 already exists 2020-04-14 22:54:55 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-1 already exists 2020-04-14 22:54:55 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-2 already exists 2020-04-14 22:54:55 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 22:54:55 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 22:54:55 DEBUG SecretOperator:168 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:54:55 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been deleted 2020-04-14 22:54:55 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:54:55 DEBUG ZookeeperSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/resources"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/1/configMap/defaultMode"} 2020-04-14 22:54:55 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/status"} 2020-04-14 22:54:55 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-zookeeper 2020-04-14 22:54:55 DEBUG ZookeeperSetOperator:168 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 22:54:55 DEBUG ZookeeperSetOperator:74 - Considering rolling update of kafka-cluster-test/my-cluster-zookeeper 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-0. 2020-04-14 22:54:55 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-1. 2020-04-14 22:54:55 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-2. 2020-04-14 22:54:55 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:55 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 22:54:55 DEBUG Util:84 - Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test is ready 2020-04-14 22:54:55 DEBUG Util:84 - Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test is ready 2020-04-14 22:54:55 DEBUG KafkaAssemblyOperator:1538 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Verifying that Zookeeper is configured to run with 3 replicas 2020-04-14 22:54:55 DEBUG ZookeeperScaler:62 - Creating Zookeeper Scaler for cluster my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:54:55 DEBUG Util:62 - Waiting for ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 to get ready 2020-04-14 22:54:55 DEBUG SslHandler:1743 - [id: 0x3dac76eb, L:/172.17.0.6:38546 - R:my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc/172.17.0.14:2181] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 2020-04-14 22:54:55 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:SyncConnected type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:54:56 DEBUG Util:84 - ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 is ready 2020-04-14 22:54:56 DEBUG ZookeeperScaler:198 - Current Zookeeper configuration is {server.2=127.0.0.1:28881:38881:participant;127.0.0.1:21811, server.1=127.0.0.1:28880:38880:participant;127.0.0.1:21810, server.3=127.0.0.1:28882:38882:participant;127.0.0.1:21812} 2020-04-14 22:54:56 DEBUG ZookeeperScaler:181 - The Zookeeper server configuration is already up to date 2020-04-14 22:54:56 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:Closed type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 22:54:56 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test to get ready 2020-04-14 22:54:56 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test is ready 2020-04-14 22:54:56 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test to get ready 2020-04-14 22:54:56 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test is ready 2020-04-14 22:54:56 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-kafka 2020-04-14 22:54:56 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-kafka already exists, patching it 2020-04-14 22:54:56 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-kafka in namespace kafka-cluster-test has been patched 2020-04-14 22:54:56 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:56 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:56 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 22:54:56 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 22:54:56 DEBUG KafkaAssemblyOperator:920 - Kafka.spec.kafka.version unchanged 2020-04-14 22:54:56 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 22:54:56 DEBUG ClusterRoleBindingOperator:110 - ClusterRoleBinding strimzi-kafka-cluster-test-my-cluster-kafka-init does not exist, noop 2020-04-14 22:54:56 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-bootstrap already exists, patching it 2020-04-14 22:54:56 DEBUG ServiceOperator:168 - Service my-cluster-kafka-bootstrap in namespace kafka-cluster-test has been patched 2020-04-14 22:54:56 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 22:54:56 DEBUG ServiceOperator:168 - Service my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 22:54:56 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-external-bootstrap does not exist, noop 2020-04-14 22:54:56 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 22:54:56 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 22:54:56 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 22:54:56 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, noop 2020-04-14 22:54:56 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 22:54:56 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 22:54:56 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 22:54:56 DEBUG AbstractModel:739 - Generating certificates 2020-04-14 22:54:56 DEBUG Ca:193 - cluster-ca: Reconciling kafka broker certificates 2020-04-14 22:54:56 DEBUG Ca:361 - Certificate for my-cluster-kafka-0 already exists 2020-04-14 22:54:56 DEBUG Ca:361 - Certificate for my-cluster-kafka-1 already exists 2020-04-14 22:54:56 DEBUG Ca:361 - Certificate for my-cluster-kafka-2 already exists 2020-04-14 22:54:56 DEBUG AbstractModel:747 - End generating certificates 2020-04-14 22:54:56 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-kafka-config already exists, patching it 2020-04-14 22:54:56 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-kafka-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:54:56 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 22:54:56 DEBUG SecretOperator:168 - Secret my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 22:54:56 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-jmx does not exist, noop 2020-04-14 22:54:56 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 22:54:56 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been deleted 2020-04-14 22:54:56 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 22:54:56 DEBUG KafkaSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/resources"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/4/configMap/defaultMode"} 2020-04-14 22:54:56 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/status"} 2020-04-14 22:54:56 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-kafka 2020-04-14 22:54:56 DEBUG KafkaSetOperator:168 - StatefulSet my-cluster-kafka in namespace kafka-cluster-test has been patched 2020-04-14 22:54:57 DEBUG KafkaRoller:150 - Initial order for rolling restart [0, 1, 2] 2020-04-14 22:54:57 DEBUG KafkaRoller:194 - Considering restart of pod 0 after delay of 0 MILLISECONDS 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:57 DEBUG KafkaRoller:281 - Pod 0 does not need to be restarted 2020-04-14 22:54:57 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 0 to become ready 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG KafkaRoller:284 - Pod 0 is now ready 2020-04-14 22:54:57 DEBUG KafkaRoller:194 - Considering restart of pod 1 after delay of 0 MILLISECONDS 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:57 DEBUG KafkaRoller:281 - Pod 1 does not need to be restarted 2020-04-14 22:54:57 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 1 to become ready 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG KafkaRoller:284 - Pod 1 is now ready 2020-04-14 22:54:57 DEBUG KafkaRoller:194 - Considering restart of pod 2 after delay of 0 MILLISECONDS 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 22:54:57 DEBUG KafkaRoller:281 - Pod 2 does not need to be restarted 2020-04-14 22:54:57 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 2 to become ready 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG KafkaRoller:284 - Pod 2 is now ready 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-0. 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-1. 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-2. 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG Util:84 - Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG Util:84 - Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 22:54:57 DEBUG RoleBindingOperator:111 - RoleBinding kafka-cluster-test/strimzi-my-cluster-topic-operator does not exist, noop 2020-04-14 22:54:57 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-topic-operator-config does not exist, noop 2020-04-14 22:54:57 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-topic-operator-certs does not exist, noop 2020-04-14 22:54:57 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 22:54:57 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-entity-operator already exists, patching it 2020-04-14 22:54:57 DEBUG RoleBindingOperator:102 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-topic-operator already exists, patching it 2020-04-14 22:54:57 DEBUG RoleBindingOperator:168 - RoleBinding strimzi-my-cluster-entity-topic-operator in namespace kafka-cluster-test has been patched 2020-04-14 22:54:57 DEBUG RoleBindingOperator:102 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-user-operator already exists, patching it 2020-04-14 22:54:57 DEBUG RoleBindingOperator:168 - RoleBinding strimzi-my-cluster-entity-user-operator in namespace kafka-cluster-test has been patched 2020-04-14 22:54:57 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-entity-topic-operator-config already exists, patching it 2020-04-14 22:54:57 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-entity-topic-operator-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:54:57 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-entity-user-operator-config already exists, patching it 2020-04-14 22:54:57 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-entity-user-operator-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 22:54:57 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-entity-operator-certs already exists, patching it 2020-04-14 22:54:57 DEBUG SecretOperator:168 - Secret my-cluster-entity-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 22:54:57 DEBUG DeploymentOperator:102 - Deployment kafka-cluster-test/my-cluster-entity-operator already exists, patching it 2020-04-14 22:54:57 DEBUG DeploymentOperator:168 - Deployment my-cluster-entity-operator in namespace kafka-cluster-test has been patched 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 22:54:57 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 22:54:57 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 22:54:57 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-exporter-certs does not exist, noop 2020-04-14 22:54:57 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 22:54:57 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 22:54:57 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 22:54:57 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-jmxtrans-config does not exist, noop 2020-04-14 22:54:57 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 22:54:57 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2020-04-14T22:54:57+0000"} 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:474 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Status did not change 2020-04-14 22:54:57 DEBUG KafkaAssemblyOperator:229 - Status for my-cluster is up to date 2020-04-14 22:54:57 INFO AbstractOperator:317 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): reconciled 2020-04-14 22:54:57 DEBUG AbstractOperator:237 - Reconciliation #18(timer) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 22:54:58 DEBUG PoolThreadCache:250 - Freed 16 thread-local buffer(s) from thread: epollEventLoopGroup-7-1 2020-04-14 22:55:07 INFO OperatorWatcher:40 - Reconciliation #19(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was DELETED 2020-04-14 22:55:07 DEBUG AbstractOperator:226 - Reconciliation #19(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 22:55:07 INFO AbstractOperator:179 - Reconciliation #19(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be deleted 2020-04-14 22:55:07 INFO AbstractOperator:184 - Reconciliation #19(watch) Kafka(kafka-cluster-test/my-cluster): Assembly my-cluster should be deleted by garbage collection 2020-04-14 22:55:07 INFO AbstractOperator:317 - Reconciliation #19(watch) Kafka(kafka-cluster-test/my-cluster): reconciled 2020-04-14 22:55:07 DEBUG AbstractOperator:237 - Reconciliation #19(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 22:55:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:55:48 INFO OperatorWatcher:40 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was ADDED 2020-04-14 22:55:48 DEBUG AbstractOperator:226 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 22:55:48 INFO AbstractOperator:173 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 22:55:48 DEBUG KafkaAssemblyOperator:504 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Setting the initial status for a new resource 2020-04-14 22:55:48 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/","value":{"conditions":[{"type":"NotReady","status":"True","lastTransitionTime":"2020-04-14T22:55:48+0000","reason":"Creating","message":"Kafka cluster is being deployed"}],"observedGeneration":0}} 2020-04-14 22:55:48 DEBUG StatusDiff:43 - Current Status path / has value 2020-04-14 22:55:48 DEBUG StatusDiff:44 - Desired Status path / has value 2020-04-14 22:55:49 INFO OperatorWatcher:40 - Reconciliation #21(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was MODIFIED 2020-04-14 22:55:49 DEBUG KafkaAssemblyOperator:466 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Completed status update 2020-04-14 22:55:49 DEBUG Ca:646 - cluster-ca: CA key (in my-cluster-cluster-ca) and certificate (in my-cluster-cluster-ca-cert) needs to be created: CA key secret my-cluster-cluster-ca is missing or lacking data.ca\.key 2020-04-14 22:55:49 DEBUG Ca:512 - cluster-ca renewalType CREATE 2020-04-14 22:55:49 DEBUG Ca:886 - Generating CA with subject=/O=io.strimzi/CN=cluster-ca v0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -x509, -new, -days, 365, -batch, -nodes, -out, /tmp/tls2978203342050434248cluster-ca v0-cert, -keyout, /tmp/tls8978801949365119360cluster-ca v0-key, -subj, /O=io.strimzi/CN=cluster-ca v0] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ..............................................................................................................+++ ......................+++ unable to write 'random state' writing new private key to '/tmp/tls8978801949365119360cluster-ca v0-key' ----- 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG Ca:552 - cluster-ca: CA key (in my-cluster-cluster-ca) and certificate (in my-cluster-cluster-ca-cert) created 2020-04-14 22:55:49 DEBUG Ca:646 - clients-ca: CA key (in my-cluster-clients-ca) and certificate (in my-cluster-clients-ca-cert) needs to be created: CA key secret my-cluster-clients-ca is missing or lacking data.ca\.key 2020-04-14 22:55:49 DEBUG Ca:512 - clients-ca renewalType CREATE 2020-04-14 22:55:49 DEBUG Ca:886 - Generating CA with subject=/O=io.strimzi/CN=clients-ca v0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -x509, -new, -days, 365, -batch, -nodes, -out, /tmp/tls6651569767365708512clients-ca v0-cert, -keyout, /tmp/tls8081127872910890527clients-ca v0-key, -subj, /O=io.strimzi/CN=clients-ca v0] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ..+++ ...................................+++ unable to write 'random state' writing new private key to '/tmp/tls8081127872910890527clients-ca v0-key' ----- 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG Ca:552 - clients-ca: CA key (in my-cluster-clients-ca) and certificate (in my-cluster-clients-ca-cert) created 2020-04-14 22:55:49 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert does not exist, creating it 2020-04-14 22:55:49 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-clients-ca-cert does not exist, creating it 2020-04-14 22:55:49 DEBUG SecretOperator:195 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG SecretOperator:195 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-ca does not exist, creating it 2020-04-14 22:55:49 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-clients-ca does not exist, creating it 2020-04-14 22:55:49 DEBUG SecretOperator:195 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG SecretOperator:195 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG ModelUtils:244 - Certificate for pod cluster-operator need to be regenerated because: certificate doesn't exist yet 2020-04-14 22:55:49 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=cluster-operator with SAN null, signed by CA cluster-ca 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls7143311161845663358key, -out, /tmp/tls6428408747622987451csr, -subj, /O=io.strimzi/CN=cluster-operator] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ...................................................................+++ ................................+++ unable to write 'random state' writing new private key to '/tmp/tls7143311161845663358key' ----- 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls6428408747622987451csr, -CA, /tmp/ca-crt-4117823468632614232.crt, -CAkey, /tmp/ca-key-5151328381508264720.key, -CAcreateserial, -out, /tmp/tls5259228513949752015cert] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=cluster-operator Getting CA Private Key unable to write 'random state' 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls5259228513949752015cert, -inkey, /tmp/tls7143311161845663358key, -name, cluster-operator, -out, /tmp/tls8434181991201076069p12, -passout, pass:J9ns4tsPHols] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG ModelUtils:252 - End generating certificates 2020-04-14 22:55:49 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs does not exist, creating it 2020-04-14 22:55:49 DEBUG SecretOperator:195 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG NetworkPolicyOperator:99 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper does not exist, creating it 2020-04-14 22:55:49 DEBUG NetworkPolicyOperator:195 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG ServiceAccountOperator:99 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 22:55:49 DEBUG ServiceAccountOperator:195 - ServiceAccount my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-zookeeper-client does not exist, creating it 2020-04-14 22:55:49 DEBUG ServiceOperator:195 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-zookeeper-nodes does not exist, creating it 2020-04-14 22:55:49 DEBUG ServiceOperator:195 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config does not exist, creating it 2020-04-14 22:55:49 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has been created 2020-04-14 22:55:49 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 22:55:49 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 22:55:49 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 22:55:49 DEBUG Ca:410 - Certificate for my-cluster-zookeeper-0 to generate 2020-04-14 22:55:49 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-zookeeper with SAN {DNS.5=my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local, DNS.6=my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc, DNS.3=my-cluster-zookeeper-client.kafka-cluster-test.svc, DNS.4=my-cluster-zookeeper-client.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-zookeeper-client, DNS.2=my-cluster-zookeeper-client.kafka-cluster-test}, signed by CA cluster-ca 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls763773213536500003broker-key, -out, /tmp/tls7805119862192292394broker-csr, -config, /tmp/sna-1218203487873391689.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-zookeeper] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .+++ .................+++ unable to write 'random state' writing new private key to '/tmp/tls763773213536500003broker-key' ----- 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls7805119862192292394broker-csr, -CA, /tmp/ca-crt-3721235822125913550.crt, -CAkey, /tmp/ca-key-8616123826851531303.key, -CAcreateserial, -out, /tmp/tls4647487606188020108broker-cert, -extfile, /tmp/sna-8812272533660704799.conf, -extensions, v3_req] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-zookeeper Getting CA Private Key unable to write 'random state' 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls4647487606188020108broker-cert, -inkey, /tmp/tls763773213536500003broker-key, -name, my-cluster-zookeeper, -out, /tmp/tls5709850883361331551broker-p12, -passout, pass:zP9t6nqJeaEA] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG Ca:410 - Certificate for my-cluster-zookeeper-1 to generate 2020-04-14 22:55:49 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-zookeeper with SAN {DNS.5=my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local, DNS.6=my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc, DNS.3=my-cluster-zookeeper-client.kafka-cluster-test.svc, DNS.4=my-cluster-zookeeper-client.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-zookeeper-client, DNS.2=my-cluster-zookeeper-client.kafka-cluster-test}, signed by CA cluster-ca 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls763773213536500003broker-key, -out, /tmp/tls7805119862192292394broker-csr, -config, /tmp/sna-8588900393521906175.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-zookeeper] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .................................................................................................+++ .........+++ unable to write 'random state' writing new private key to '/tmp/tls763773213536500003broker-key' ----- 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls7805119862192292394broker-csr, -CA, /tmp/ca-crt-9175997928974911571.crt, -CAkey, /tmp/ca-key-3675898302038685871.key, -CAcreateserial, -out, /tmp/tls4647487606188020108broker-cert, -extfile, /tmp/sna-7723318725389587208.conf, -extensions, v3_req] 2020-04-14 22:55:49 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-zookeeper Getting CA Private Key unable to write 'random state' 2020-04-14 22:55:49 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:49 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls4647487606188020108broker-cert, -inkey, /tmp/tls763773213536500003broker-key, -name, my-cluster-zookeeper, -out, /tmp/tls5709850883361331551broker-p12, -passout, pass:p8KcMgCqkMit] 2020-04-14 22:55:50 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:55:50 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:50 DEBUG Ca:410 - Certificate for my-cluster-zookeeper-2 to generate 2020-04-14 22:55:50 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-zookeeper with SAN {DNS.5=my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local, DNS.6=my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc, DNS.3=my-cluster-zookeeper-client.kafka-cluster-test.svc, DNS.4=my-cluster-zookeeper-client.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-zookeeper-client, DNS.2=my-cluster-zookeeper-client.kafka-cluster-test}, signed by CA cluster-ca 2020-04-14 22:55:50 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls763773213536500003broker-key, -out, /tmp/tls7805119862192292394broker-csr, -config, /tmp/sna-6761705599641120987.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-zookeeper] 2020-04-14 22:55:50 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ........................................+++ .......+++ unable to write 'random state' writing new private key to '/tmp/tls763773213536500003broker-key' ----- 2020-04-14 22:55:50 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:50 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls7805119862192292394broker-csr, -CA, /tmp/ca-crt-3407967753329914829.crt, -CAkey, /tmp/ca-key-5064129256027999541.key, -CAcreateserial, -out, /tmp/tls4647487606188020108broker-cert, -extfile, /tmp/sna-4056267664977613279.conf, -extensions, v3_req] 2020-04-14 22:55:50 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-zookeeper Getting CA Private Key unable to write 'random state' 2020-04-14 22:55:50 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:50 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls4647487606188020108broker-cert, -inkey, /tmp/tls763773213536500003broker-key, -name, my-cluster-zookeeper, -out, /tmp/tls5709850883361331551broker-p12, -passout, pass:fPuVMw7wzlNs] 2020-04-14 22:55:50 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 22:55:50 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 22:55:50 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 22:55:50 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes does not exist, creating it 2020-04-14 22:55:50 DEBUG SecretOperator:195 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been created 2020-04-14 22:55:50 DEBUG PodDisruptionBudgetOperator:99 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 22:55:50 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:55:50 DEBUG ZookeeperSetOperator:99 - StatefulSet kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 22:55:50 DEBUG ZookeeperSetOperator:195 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 22:55:50 DEBUG Util:62 - Waiting for StatefulSet resource my-cluster-zookeeper in namespace kafka-cluster-test to get ready 2020-04-14 22:55:51 DEBUG Util:84 - StatefulSet resource my-cluster-zookeeper in namespace kafka-cluster-test is ready 2020-04-14 22:55:51 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 22:55:51 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test to get ready 2020-04-14 22:55:51 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test to get ready 2020-04-14 22:56:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:57:05 WARN AbstractOperator:246 - Reconciliation #24(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:57:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:57:35 WARN AbstractOperator:246 - Reconciliation #25(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:57:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:58:05 WARN AbstractOperator:246 - Reconciliation #26(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:58:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:58:35 WARN AbstractOperator:246 - Reconciliation #27(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:58:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:59:03 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 22:59:05 WARN AbstractOperator:246 - Reconciliation #28(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:59:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 22:59:35 WARN AbstractOperator:246 - Reconciliation #29(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 22:59:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:00:05 WARN AbstractOperator:246 - Reconciliation #30(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:00:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:00:35 WARN AbstractOperator:246 - Reconciliation #31(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:00:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:01:05 WARN AbstractOperator:246 - Reconciliation #32(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:01:19 ERROR Util:90 - Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test to be ready 2020-04-14 23:01:19 ERROR Util:90 - Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to be ready 2020-04-14 23:01:19 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2020-04-14T23:01:19+0000"} 2020-04-14 23:01:19 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/reason","value":"TimeoutException"} 2020-04-14 23:01:19 DEBUG StatusDiff:43 - Current Status path /conditions/0/reason has value "Creating" 2020-04-14 23:01:19 DEBUG StatusDiff:44 - Desired Status path /conditions/0/reason has value "TimeoutException" 2020-04-14 23:01:19 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/message","value":"Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to be ready"} 2020-04-14 23:01:19 DEBUG StatusDiff:43 - Current Status path /conditions/0/message has value "Kafka cluster is being deployed" 2020-04-14 23:01:19 DEBUG StatusDiff:44 - Desired Status path /conditions/0/message has value "Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to be ready" 2020-04-14 23:01:19 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/observedGeneration","value":1} 2020-04-14 23:01:19 DEBUG StatusDiff:43 - Current Status path /observedGeneration has value 0 2020-04-14 23:01:19 DEBUG StatusDiff:44 - Desired Status path /observedGeneration has value 1 2020-04-14 23:01:19 DEBUG StatusDiff:42 - Status differs: {"op":"add","path":"/listeners","value":[{"type":"plain","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9092}]},{"type":"tls","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9093}]}]} 2020-04-14 23:01:19 DEBUG StatusDiff:43 - Current Status path /listeners has value 2020-04-14 23:01:19 DEBUG StatusDiff:44 - Desired Status path /listeners has value [{"type":"plain","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9092}]},{"type":"tls","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9093}]}] 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:466 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Completed status update 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:229 - Status for my-cluster is up to date 2020-04-14 23:01:19 ERROR AbstractOperator:175 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): createOrUpdate failed io.strimzi.operator.common.operator.resource.TimeoutException: Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to be ready at io.strimzi.operator.common.Util$1.lambda$handle$1(Util.java:91) ~[io.strimzi.operator-common-0.18.0-SNAPSHOT.jar:0.18.0-SNAPSHOT] at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:330) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [io.netty.netty-transport-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242] 2020-04-14 23:01:19 INFO OperatorWatcher:40 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was MODIFIED 2020-04-14 23:01:19 WARN AbstractOperator:329 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Failed to reconcile io.strimzi.operator.common.operator.resource.TimeoutException: Exceeded timeout of 300000ms while waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to be ready at io.strimzi.operator.common.Util$1.lambda$handle$1(Util.java:91) ~[io.strimzi.operator-common-0.18.0-SNAPSHOT.jar:0.18.0-SNAPSHOT] at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:330) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38) ~[io.vertx.vertx-core-3.8.5.jar:3.8.5] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [io.netty.netty-transport-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.45.Final.jar:4.1.45.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242] 2020-04-14 23:01:19 DEBUG AbstractOperator:237 - Reconciliation #20(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 23:01:19 DEBUG AbstractOperator:226 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 23:01:19 INFO AbstractOperator:173 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:520 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Status is already set. No need to set initial status 2020-04-14 23:01:19 DEBUG Ca:653 - cluster-ca: The CA certificate in secret my-cluster-cluster-ca-cert already exists and does not need renewing 2020-04-14 23:01:19 DEBUG Ca:512 - cluster-ca renewalType NOOP 2020-04-14 23:01:19 DEBUG Ca:653 - clients-ca: The CA certificate in secret my-cluster-clients-ca-cert already exists and does not need renewing 2020-04-14 23:01:19 DEBUG Ca:512 - clients-ca renewalType NOOP 2020-04-14 23:01:19 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert already exists, patching it 2020-04-14 23:01:19 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca-cert already exists, patching it 2020-04-14 23:01:19 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca already exists, patching it 2020-04-14 23:01:19 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca already exists, patching it 2020-04-14 23:01:19 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs already exists, patching it 2020-04-14 23:01:19 DEBUG SecretOperator:168 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-zookeeper 2020-04-14 23:01:19 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper already exists, patching it 2020-04-14 23:01:19 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:783 - Kafka.spec.kafka.version is unchanged therefore no change to Zookeeper is required 2020-04-14 23:01:19 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:01:19 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-client already exists, patching it 2020-04-14 23:01:19 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 23:01:19 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config already exists, patching it 2020-04-14 23:01:19 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:01:19 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 23:01:19 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 23:01:19 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 23:01:19 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-0 already exists 2020-04-14 23:01:19 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-1 already exists 2020-04-14 23:01:19 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-2 already exists 2020-04-14 23:01:19 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 23:01:19 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 23:01:19 DEBUG SecretOperator:168 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:01:19 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been deleted 2020-04-14 23:01:19 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 23:01:19 DEBUG ZookeeperSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/resources"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/1/configMap/defaultMode"} 2020-04-14 23:01:19 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/status"} 2020-04-14 23:01:19 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-zookeeper 2020-04-14 23:01:19 DEBUG ZookeeperSetOperator:168 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 23:01:19 DEBUG ZookeeperSetOperator:74 - Considering rolling update of kafka-cluster-test/my-cluster-zookeeper 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-1 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-2 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-0. 2020-04-14 23:01:19 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-1. 2020-04-14 23:01:19 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test to get ready 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-2. 2020-04-14 23:01:19 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test to get ready 2020-04-14 23:01:19 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 23:01:19 DEBUG Util:84 - Pods resource my-cluster-zookeeper-1 in namespace kafka-cluster-test is ready 2020-04-14 23:01:19 DEBUG Util:84 - Pods resource my-cluster-zookeeper-2 in namespace kafka-cluster-test is ready 2020-04-14 23:01:19 DEBUG KafkaAssemblyOperator:1538 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Verifying that Zookeeper is configured to run with 3 replicas 2020-04-14 23:01:19 DEBUG ZookeeperScaler:62 - Creating Zookeeper Scaler for cluster my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:01:19 DEBUG Util:62 - Waiting for ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 to get ready 2020-04-14 23:01:20 DEBUG SslHandler:1743 - [id: 0xdbf810b2, L:/172.17.0.6:40716 - R:my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc/172.17.0.14:2181] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 2020-04-14 23:01:20 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:SyncConnected type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:01:20 DEBUG Util:84 - ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 is ready 2020-04-14 23:01:20 DEBUG ZookeeperScaler:198 - Current Zookeeper configuration is {server.2=127.0.0.1:28881:38881:participant;127.0.0.1:21811, server.1=127.0.0.1:28880:38880:participant;127.0.0.1:21810, server.3=127.0.0.1:28882:38882:participant;127.0.0.1:21812} 2020-04-14 23:01:20 DEBUG ZookeeperScaler:181 - The Zookeeper server configuration is already up to date 2020-04-14 23:01:20 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:Closed type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-1.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181,my-cluster-zookeeper-2.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:01:20 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test to get ready 2020-04-14 23:01:20 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test is ready 2020-04-14 23:01:20 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test to get ready 2020-04-14 23:01:20 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test is ready 2020-04-14 23:01:20 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-kafka 2020-04-14 23:01:20 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-kafka already exists, patching it 2020-04-14 23:01:20 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-kafka in namespace kafka-cluster-test has been patched 2020-04-14 23:01:20 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:01:20 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:01:20 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:01:20 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 23:01:20 DEBUG KafkaAssemblyOperator:920 - Kafka.spec.kafka.version unchanged 2020-04-14 23:01:20 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:01:20 DEBUG ClusterRoleBindingOperator:110 - ClusterRoleBinding strimzi-kafka-cluster-test-my-cluster-kafka-init does not exist, noop 2020-04-14 23:01:20 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-bootstrap already exists, patching it 2020-04-14 23:01:20 DEBUG ServiceOperator:168 - Service my-cluster-kafka-bootstrap in namespace kafka-cluster-test has been patched 2020-04-14 23:01:20 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 23:01:20 DEBUG ServiceOperator:168 - Service my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 23:01:20 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-external-bootstrap does not exist, noop 2020-04-14 23:01:20 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:01:20 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 23:01:20 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 23:01:20 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, noop 2020-04-14 23:01:20 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-1 does not exist, noop 2020-04-14 23:01:20 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-2 does not exist, noop 2020-04-14 23:01:20 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:01:20 DEBUG AbstractModel:739 - Generating certificates 2020-04-14 23:01:20 DEBUG Ca:193 - cluster-ca: Reconciling kafka broker certificates 2020-04-14 23:01:20 DEBUG Ca:361 - Certificate for my-cluster-kafka-0 already exists 2020-04-14 23:01:20 DEBUG Ca:361 - Certificate for my-cluster-kafka-1 already exists 2020-04-14 23:01:20 DEBUG Ca:361 - Certificate for my-cluster-kafka-2 already exists 2020-04-14 23:01:20 DEBUG AbstractModel:747 - End generating certificates 2020-04-14 23:01:20 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-kafka-config already exists, patching it 2020-04-14 23:01:20 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-kafka-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:01:20 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 23:01:20 DEBUG SecretOperator:168 - Secret my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 23:01:20 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-jmx does not exist, noop 2020-04-14 23:01:20 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:01:20 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been deleted 2020-04-14 23:01:20 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 23:01:20 DEBUG KafkaSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/resources"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/4/configMap/defaultMode"} 2020-04-14 23:01:20 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/status"} 2020-04-14 23:01:20 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-kafka 2020-04-14 23:01:20 DEBUG KafkaSetOperator:168 - StatefulSet my-cluster-kafka in namespace kafka-cluster-test has been patched 2020-04-14 23:01:21 DEBUG KafkaRoller:150 - Initial order for rolling restart [2, 1, 0] 2020-04-14 23:01:21 DEBUG KafkaRoller:194 - Considering restart of pod 2 after delay of 0 MILLISECONDS 2020-04-14 23:01:21 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:01:21 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:01:21 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-2 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:01:21 DEBUG KafkaRoller:281 - Pod 2 does not need to be restarted 2020-04-14 23:01:21 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 2 to become ready 2020-04-14 23:01:21 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test to get ready 2020-04-14 23:01:22 DEBUG PoolThreadCache:250 - Freed 16 thread-local buffer(s) from thread: epollEventLoopGroup-9-1 2020-04-14 23:01:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:01:35 WARN AbstractOperator:246 - Reconciliation #34(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:01:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:02:05 WARN AbstractOperator:246 - Reconciliation #35(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:02:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:02:35 WARN AbstractOperator:246 - Reconciliation #36(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:02:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:03:05 WARN AbstractOperator:246 - Reconciliation #37(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:03:24 DEBUG Util:84 - Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test is ready 2020-04-14 23:03:24 DEBUG KafkaRoller:284 - Pod 2 is now ready 2020-04-14 23:03:24 DEBUG KafkaRoller:194 - Considering restart of pod 1 after delay of 0 MILLISECONDS 2020-04-14 23:03:24 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:03:24 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:03:24 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-1 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:03:24 DEBUG KafkaRoller:281 - Pod 1 does not need to be restarted 2020-04-14 23:03:24 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 1 to become ready 2020-04-14 23:03:24 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to get ready 2020-04-14 23:03:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:03:35 WARN AbstractOperator:246 - Reconciliation #38(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:03:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:04:05 WARN AbstractOperator:246 - Reconciliation #39(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:04:22 DEBUG Util:84 - Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test is ready 2020-04-14 23:04:22 DEBUG KafkaRoller:284 - Pod 1 is now ready 2020-04-14 23:04:22 DEBUG KafkaRoller:194 - Considering restart of pod 0 after delay of 0 MILLISECONDS 2020-04-14 23:04:22 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:04:22 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:04:22 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:04:22 DEBUG KafkaRoller:281 - Pod 0 does not need to be restarted 2020-04-14 23:04:22 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 0 to become ready 2020-04-14 23:04:22 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:04:22 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:04:22 DEBUG KafkaRoller:284 - Pod 0 is now ready 2020-04-14 23:04:22 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-0. 2020-04-14 23:04:22 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:04:22 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-1. 2020-04-14 23:04:22 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test to get ready 2020-04-14 23:04:22 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #33(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-2. 2020-04-14 23:04:22 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test to get ready 2020-04-14 23:04:22 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:04:22 DEBUG Util:84 - Pods resource my-cluster-kafka-1 in namespace kafka-cluster-test is ready 2020-04-14 23:04:22 DEBUG Util:84 - Pods resource my-cluster-kafka-2 in namespace kafka-cluster-test is ready 2020-04-14 23:04:22 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test to get ready 2020-04-14 23:04:22 DEBUG Util:84 - Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test is ready 2020-04-14 23:04:22 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test to get ready 2020-04-14 23:04:22 DEBUG Util:84 - Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test is ready 2020-04-14 23:04:22 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:04:22 DEBUG RoleBindingOperator:111 - RoleBinding kafka-cluster-test/strimzi-my-cluster-topic-operator does not exist, noop 2020-04-14 23:04:22 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-topic-operator-config does not exist, noop 2020-04-14 23:04:22 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-topic-operator-certs does not exist, noop 2020-04-14 23:04:22 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:04:22 DEBUG ServiceAccountOperator:99 - ServiceAccount kafka-cluster-test/my-cluster-entity-operator does not exist, creating it 2020-04-14 23:04:22 DEBUG ServiceAccountOperator:195 - ServiceAccount my-cluster-entity-operator in namespace kafka-cluster-test has been created 2020-04-14 23:04:22 DEBUG RoleBindingOperator:99 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-topic-operator does not exist, creating it 2020-04-14 23:04:22 DEBUG RoleBindingOperator:195 - RoleBinding strimzi-my-cluster-entity-topic-operator in namespace kafka-cluster-test has been created 2020-04-14 23:04:22 DEBUG RoleBindingOperator:99 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-user-operator does not exist, creating it 2020-04-14 23:04:22 DEBUG RoleBindingOperator:195 - RoleBinding strimzi-my-cluster-entity-user-operator in namespace kafka-cluster-test has been created 2020-04-14 23:04:22 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-entity-topic-operator-config does not exist, creating it 2020-04-14 23:04:22 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-entity-topic-operator-config in namespace kafka-cluster-test has been created 2020-04-14 23:04:22 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-entity-user-operator-config does not exist, creating it 2020-04-14 23:04:22 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-entity-user-operator-config in namespace kafka-cluster-test has been created 2020-04-14 23:04:22 DEBUG ModelUtils:244 - Certificate for pod entity-operator need to be regenerated because: certificate doesn't exist yet 2020-04-14 23:04:22 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-entity-operator with SAN null, signed by CA cluster-ca 2020-04-14 23:04:22 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls3387020688338671597key, -out, /tmp/tls2130368355049208519csr, -subj, /O=io.strimzi/CN=my-cluster-entity-operator] 2020-04-14 23:04:22 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ...............+++ ...................+++ unable to write 'random state' writing new private key to '/tmp/tls3387020688338671597key' ----- 2020-04-14 23:04:22 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:04:22 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls2130368355049208519csr, -CA, /tmp/ca-crt-1994814732192964710.crt, -CAkey, /tmp/ca-key-2543031884990543778.key, -CAcreateserial, -out, /tmp/tls6572061087592283735cert] 2020-04-14 23:04:22 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-entity-operator Getting CA Private Key unable to write 'random state' 2020-04-14 23:04:22 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:04:22 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls6572061087592283735cert, -inkey, /tmp/tls3387020688338671597key, -name, my-cluster-entity-operator, -out, /tmp/tls1300204764534302727p12, -passout, pass:DRllJ6FKxCz3] 2020-04-14 23:04:22 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 23:04:22 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:04:22 DEBUG ModelUtils:252 - End generating certificates 2020-04-14 23:04:22 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-entity-operator-certs does not exist, creating it 2020-04-14 23:04:22 DEBUG SecretOperator:195 - Secret my-cluster-entity-operator-certs in namespace kafka-cluster-test has been created 2020-04-14 23:04:22 DEBUG DeploymentOperator:99 - Deployment kafka-cluster-test/my-cluster-entity-operator does not exist, creating it 2020-04-14 23:04:22 DEBUG DeploymentOperator:195 - Deployment my-cluster-entity-operator in namespace kafka-cluster-test has been created 2020-04-14 23:04:22 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:04:23 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 23:04:23 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:04:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:04:53 DEBUG PoolThreadCache:250 - Freed 16 thread-local buffer(s) from thread: epollEventLoopGroup-12-1 2020-04-14 23:04:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:05:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:05:38 INFO OperatorWatcher:40 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was ADDED 2020-04-14 23:05:38 DEBUG AbstractOperator:226 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 23:05:38 INFO AbstractOperator:173 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 23:05:38 DEBUG KafkaAssemblyOperator:504 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Setting the initial status for a new resource 2020-04-14 23:05:38 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/","value":{"conditions":[{"type":"NotReady","status":"True","lastTransitionTime":"2020-04-14T23:05:38+0000","reason":"Creating","message":"Kafka cluster is being deployed"}],"observedGeneration":0}} 2020-04-14 23:05:38 DEBUG StatusDiff:43 - Current Status path / has value 2020-04-14 23:05:38 DEBUG StatusDiff:44 - Desired Status path / has value 2020-04-14 23:05:38 DEBUG KafkaAssemblyOperator:466 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Completed status update 2020-04-14 23:05:38 INFO OperatorWatcher:40 - Reconciliation #46(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was MODIFIED 2020-04-14 23:05:38 DEBUG Ca:646 - cluster-ca: CA key (in my-cluster-cluster-ca) and certificate (in my-cluster-cluster-ca-cert) needs to be created: CA key secret my-cluster-cluster-ca is missing or lacking data.ca\.key 2020-04-14 23:05:38 DEBUG Ca:512 - cluster-ca renewalType CREATE 2020-04-14 23:05:38 DEBUG Ca:886 - Generating CA with subject=/O=io.strimzi/CN=cluster-ca v0 2020-04-14 23:05:38 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -x509, -new, -days, 365, -batch, -nodes, -out, /tmp/tls3199717161500910627cluster-ca v0-cert, -keyout, /tmp/tls2233470449705628845cluster-ca v0-key, -subj, /O=io.strimzi/CN=cluster-ca v0] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ..+++ ............+++ unable to write 'random state' writing new private key to '/tmp/tls2233470449705628845cluster-ca v0-key' ----- 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG Ca:552 - cluster-ca: CA key (in my-cluster-cluster-ca) and certificate (in my-cluster-cluster-ca-cert) created 2020-04-14 23:05:39 DEBUG Ca:646 - clients-ca: CA key (in my-cluster-clients-ca) and certificate (in my-cluster-clients-ca-cert) needs to be created: CA key secret my-cluster-clients-ca is missing or lacking data.ca\.key 2020-04-14 23:05:39 DEBUG Ca:512 - clients-ca renewalType CREATE 2020-04-14 23:05:39 DEBUG Ca:886 - Generating CA with subject=/O=io.strimzi/CN=clients-ca v0 2020-04-14 23:05:39 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -x509, -new, -days, 365, -batch, -nodes, -out, /tmp/tls3453947057686894823clients-ca v0-cert, -keyout, /tmp/tls4875254937944612056clients-ca v0-key, -subj, /O=io.strimzi/CN=clients-ca v0] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .........................................................................+++ ...+++ unable to write 'random state' writing new private key to '/tmp/tls4875254937944612056clients-ca v0-key' ----- 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG Ca:552 - clients-ca: CA key (in my-cluster-clients-ca) and certificate (in my-cluster-clients-ca-cert) created 2020-04-14 23:05:39 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-clients-ca-cert does not exist, creating it 2020-04-14 23:05:39 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert does not exist, creating it 2020-04-14 23:05:39 DEBUG SecretOperator:195 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG SecretOperator:195 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-ca does not exist, creating it 2020-04-14 23:05:39 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-clients-ca does not exist, creating it 2020-04-14 23:05:39 DEBUG SecretOperator:195 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG SecretOperator:195 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG ModelUtils:244 - Certificate for pod cluster-operator need to be regenerated because: certificate doesn't exist yet 2020-04-14 23:05:39 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=cluster-operator with SAN null, signed by CA cluster-ca 2020-04-14 23:05:39 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls2401745789409569402key, -out, /tmp/tls7250967529874879018csr, -subj, /O=io.strimzi/CN=cluster-operator] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ......+++ ...........+++ unable to write 'random state' writing new private key to '/tmp/tls2401745789409569402key' ----- 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls7250967529874879018csr, -CA, /tmp/ca-crt-1887848243054477685.crt, -CAkey, /tmp/ca-key-4387689093140576318.key, -CAcreateserial, -out, /tmp/tls2605573372418889169cert] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=cluster-operator Getting CA Private Key unable to write 'random state' 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls2605573372418889169cert, -inkey, /tmp/tls2401745789409569402key, -name, cluster-operator, -out, /tmp/tls4863984085988888553p12, -passout, pass:KsqpZw0b5iof] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG ModelUtils:252 - End generating certificates 2020-04-14 23:05:39 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs does not exist, creating it 2020-04-14 23:05:39 DEBUG SecretOperator:195 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 WARN AbstractModel:214 - A ZooKeeper cluster with a single replica and ephemeral storage will be in a defective state after any restart or rolling update. It is recommended that a minimum of three replicas are used. 2020-04-14 23:05:39 DEBUG NetworkPolicyOperator:99 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper does not exist, creating it 2020-04-14 23:05:39 DEBUG NetworkPolicyOperator:195 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG ServiceAccountOperator:99 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 23:05:39 DEBUG ServiceAccountOperator:195 - ServiceAccount my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-zookeeper-client does not exist, creating it 2020-04-14 23:05:39 DEBUG ServiceOperator:195 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-zookeeper-nodes does not exist, creating it 2020-04-14 23:05:39 DEBUG ServiceOperator:195 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config does not exist, creating it 2020-04-14 23:05:39 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 23:05:39 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 23:05:39 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 23:05:39 DEBUG Ca:410 - Certificate for my-cluster-zookeeper-0 to generate 2020-04-14 23:05:39 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-zookeeper with SAN {DNS.5=my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc.cluster.local, DNS.6=my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc, DNS.3=my-cluster-zookeeper-client.kafka-cluster-test.svc, DNS.4=my-cluster-zookeeper-client.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-zookeeper-client, DNS.2=my-cluster-zookeeper-client.kafka-cluster-test}, signed by CA cluster-ca 2020-04-14 23:05:39 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls6830367164627752707broker-key, -out, /tmp/tls1038746011537455620broker-csr, -config, /tmp/sna-1867344958428227016.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-zookeeper] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key .......................................................+++ ......................+++ unable to write 'random state' writing new private key to '/tmp/tls6830367164627752707broker-key' ----- 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls1038746011537455620broker-csr, -CA, /tmp/ca-crt-3080328798248218636.crt, -CAkey, /tmp/ca-key-6126136450691759398.key, -CAcreateserial, -out, /tmp/tls3413383086628746755broker-cert, -extfile, /tmp/sna-4417478153200423937.conf, -extensions, v3_req] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-zookeeper Getting CA Private Key unable to write 'random state' 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls3413383086628746755broker-cert, -inkey, /tmp/tls6830367164627752707broker-key, -name, my-cluster-zookeeper, -out, /tmp/tls8389369454434540610broker-p12, -passout, pass:auPsO0NFVw6s] 2020-04-14 23:05:39 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 23:05:39 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:05:39 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 23:05:39 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes does not exist, creating it 2020-04-14 23:05:39 DEBUG SecretOperator:195 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG PodDisruptionBudgetOperator:99 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 23:05:39 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG ZookeeperSetOperator:99 - StatefulSet kafka-cluster-test/my-cluster-zookeeper does not exist, creating it 2020-04-14 23:05:39 DEBUG ZookeeperSetOperator:195 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 23:05:39 DEBUG Util:62 - Waiting for StatefulSet resource my-cluster-zookeeper in namespace kafka-cluster-test to get ready 2020-04-14 23:05:40 DEBUG Util:84 - StatefulSet resource my-cluster-zookeeper in namespace kafka-cluster-test is ready 2020-04-14 23:05:40 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:05:48 WARN AbstractOperator:246 - Reconciliation #46(watch) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:05:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:06:01 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 23:06:01 DEBUG ZookeeperSetOperator:74 - Considering rolling update of kafka-cluster-test/my-cluster-zookeeper 2020-04-14 23:06:01 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:06:01 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:06:01 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:06:01 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-0. 2020-04-14 23:06:01 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:06:01 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 23:06:01 DEBUG KafkaAssemblyOperator:1538 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Verifying that Zookeeper is configured to run with null replicas 2020-04-14 23:06:01 DEBUG ZookeeperScaler:62 - Creating Zookeeper Scaler for cluster my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:06:01 DEBUG Util:62 - Waiting for ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 to get ready 2020-04-14 23:06:02 DEBUG SslHandler:1743 - [id: 0x6e32539a, L:/172.17.0.6:47660 - R:my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc/172.17.0.9:2181] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 2020-04-14 23:06:02 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:SyncConnected type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:06:02 DEBUG Util:84 - ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 is ready 2020-04-14 23:06:02 DEBUG ZookeeperScaler:198 - Current Zookeeper configuration is {server.1=127.0.0.1:28880:38880:participant;127.0.0.1:21810} 2020-04-14 23:06:02 DEBUG ZookeeperScaler:181 - The Zookeeper server configuration is already up to date 2020-04-14 23:06:02 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:Closed type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:06:02 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test to get ready 2020-04-14 23:06:02 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test is ready 2020-04-14 23:06:02 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test to get ready 2020-04-14 23:06:02 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test is ready 2020-04-14 23:06:02 DEBUG NetworkPolicyOperator:99 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-kafka does not exist, creating it 2020-04-14 23:06:02 DEBUG NetworkPolicyOperator:195 - NetworkPolicy my-cluster-network-policy-kafka in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG ServiceAccountOperator:99 - ServiceAccount kafka-cluster-test/my-cluster-kafka does not exist, creating it 2020-04-14 23:06:03 DEBUG ServiceAccountOperator:195 - ServiceAccount my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG ClusterRoleBindingOperator:110 - ClusterRoleBinding strimzi-kafka-cluster-test-my-cluster-kafka-init does not exist, noop 2020-04-14 23:06:03 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, creating it 2020-04-14 23:06:03 DEBUG ServiceOperator:195 - Service my-cluster-kafka-bootstrap in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG ServiceOperator:99 - Service kafka-cluster-test/my-cluster-kafka-brokers does not exist, creating it 2020-04-14 23:06:03 DEBUG ServiceOperator:195 - Service my-cluster-kafka-brokers in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-external-bootstrap does not exist, noop 2020-04-14 23:06:03 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:06:03 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, noop 2020-04-14 23:06:03 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:06:03 DEBUG AbstractModel:739 - Generating certificates 2020-04-14 23:06:03 DEBUG Ca:193 - cluster-ca: Reconciling kafka broker certificates 2020-04-14 23:06:03 DEBUG Ca:410 - Certificate for my-cluster-kafka-0 to generate 2020-04-14 23:06:03 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-kafka with SAN {DNS.7=my-cluster-kafka-brokers.kafka-cluster-test.svc, DNS.8=my-cluster-kafka-brokers.kafka-cluster-test.svc.cluster.local, DNS.5=my-cluster-kafka-brokers, DNS.6=my-cluster-kafka-brokers.kafka-cluster-test, DNS.9=my-cluster-kafka-0.my-cluster-kafka-brokers.kafka-cluster-test.svc.cluster.local, DNS.3=my-cluster-kafka-bootstrap.kafka-cluster-test.svc, DNS.4=my-cluster-kafka-bootstrap.kafka-cluster-test.svc.cluster.local, DNS.1=my-cluster-kafka-bootstrap, DNS.2=my-cluster-kafka-bootstrap.kafka-cluster-test, DNS.10=my-cluster-kafka-0.my-cluster-kafka-brokers.kafka-cluster-test.svc}, signed by CA cluster-ca 2020-04-14 23:06:03 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls1380898792953343030broker-key, -out, /tmp/tls2574927123052917230broker-csr, -config, /tmp/sna-3130491094396522627.conf, -extensions, v3_req, -subj, /O=io.strimzi/CN=my-cluster-kafka] 2020-04-14 23:06:03 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ...............................................................................................................................+++ ................+++ unable to write 'random state' writing new private key to '/tmp/tls1380898792953343030broker-key' ----- 2020-04-14 23:06:03 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:06:03 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls2574927123052917230broker-csr, -CA, /tmp/ca-crt-8641488420387969585.crt, -CAkey, /tmp/ca-key-957149474870641937.key, -CAcreateserial, -out, /tmp/tls2572269059875847717broker-cert, -extfile, /tmp/sna-302586184810162782.conf, -extensions, v3_req] 2020-04-14 23:06:03 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-kafka Getting CA Private Key unable to write 'random state' 2020-04-14 23:06:03 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:06:03 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls2572269059875847717broker-cert, -inkey, /tmp/tls1380898792953343030broker-key, -name, my-cluster-kafka, -out, /tmp/tls2282059627865821257broker-p12, -passout, pass:DavLnWlnQj5j] 2020-04-14 23:06:03 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 23:06:03 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:06:03 DEBUG AbstractModel:747 - End generating certificates 2020-04-14 23:06:03 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-kafka-config does not exist, creating it 2020-04-14 23:06:03 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-kafka-config in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-kafka-brokers does not exist, creating it 2020-04-14 23:06:03 DEBUG SecretOperator:195 - Secret my-cluster-kafka-brokers in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-jmx does not exist, noop 2020-04-14 23:06:03 DEBUG PodDisruptionBudgetOperator:99 - PodDisruptionBudget kafka-cluster-test/my-cluster-kafka does not exist, creating it 2020-04-14 23:06:03 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG KafkaSetOperator:99 - StatefulSet kafka-cluster-test/my-cluster-kafka does not exist, creating it 2020-04-14 23:06:03 DEBUG KafkaSetOperator:195 - StatefulSet my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 23:06:03 DEBUG Util:62 - Waiting for StatefulSet resource my-cluster-kafka in namespace kafka-cluster-test to get ready 2020-04-14 23:06:04 DEBUG Util:84 - StatefulSet resource my-cluster-kafka in namespace kafka-cluster-test is ready 2020-04-14 23:06:04 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:06:04 DEBUG PoolThreadCache:250 - Freed 17 thread-local buffer(s) from thread: epollEventLoopGroup-13-1 2020-04-14 23:06:05 WARN AbstractOperator:246 - Reconciliation #47(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:06:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:06:35 WARN AbstractOperator:246 - Reconciliation #48(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:06:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:07:05 WARN AbstractOperator:246 - Reconciliation #49(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:07:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:07:35 WARN AbstractOperator:246 - Reconciliation #50(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:07:55 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:08:05 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:05 DEBUG KafkaRoller:150 - Initial order for rolling restart [0] 2020-04-14 23:08:05 DEBUG KafkaRoller:194 - Considering restart of pod 0 after delay of 0 MILLISECONDS 2020-04-14 23:08:05 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:08:05 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:08:05 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:08:05 DEBUG KafkaRoller:281 - Pod 0 does not need to be restarted 2020-04-14 23:08:05 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 0 to become ready 2020-04-14 23:08:05 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:05 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:05 DEBUG KafkaRoller:284 - Pod 0 is now ready 2020-04-14 23:08:05 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-0. 2020-04-14 23:08:05 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:05 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:05 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test to get ready 2020-04-14 23:08:05 DEBUG Util:84 - Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test is ready 2020-04-14 23:08:05 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test to get ready 2020-04-14 23:08:05 DEBUG Util:84 - Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test is ready 2020-04-14 23:08:05 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:05 DEBUG RoleBindingOperator:111 - RoleBinding kafka-cluster-test/strimzi-my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:05 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-topic-operator-config does not exist, noop 2020-04-14 23:08:05 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-topic-operator-certs does not exist, noop 2020-04-14 23:08:05 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:05 DEBUG AbstractModel:361 - logging is not set, using default loggers 2020-04-14 23:08:05 DEBUG AbstractModel:361 - logging is not set, using default loggers 2020-04-14 23:08:05 DEBUG ServiceAccountOperator:99 - ServiceAccount kafka-cluster-test/my-cluster-entity-operator does not exist, creating it 2020-04-14 23:08:05 DEBUG ServiceAccountOperator:195 - ServiceAccount my-cluster-entity-operator in namespace kafka-cluster-test has been created 2020-04-14 23:08:05 DEBUG RoleBindingOperator:99 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-topic-operator does not exist, creating it 2020-04-14 23:08:05 DEBUG RoleBindingOperator:195 - RoleBinding strimzi-my-cluster-entity-topic-operator in namespace kafka-cluster-test has been created 2020-04-14 23:08:05 DEBUG RoleBindingOperator:99 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-user-operator does not exist, creating it 2020-04-14 23:08:05 DEBUG RoleBindingOperator:195 - RoleBinding strimzi-my-cluster-entity-user-operator in namespace kafka-cluster-test has been created 2020-04-14 23:08:05 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-entity-topic-operator-config does not exist, creating it 2020-04-14 23:08:05 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-entity-topic-operator-config in namespace kafka-cluster-test has been created 2020-04-14 23:08:05 DEBUG ConfigMapOperator:99 - ConfigMap kafka-cluster-test/my-cluster-entity-user-operator-config does not exist, creating it 2020-04-14 23:08:05 DEBUG ConfigMapOperator:195 - ConfigMap my-cluster-entity-user-operator-config in namespace kafka-cluster-test has been created 2020-04-14 23:08:05 DEBUG ModelUtils:244 - Certificate for pod entity-operator need to be regenerated because: certificate doesn't exist yet 2020-04-14 23:08:05 DEBUG Ca:279 - Generating certificate /O=io.strimzi/CN=my-cluster-entity-operator with SAN null, signed by CA cluster-ca 2020-04-14 23:08:05 DEBUG OpenSslCertManager:383 - Running command [openssl, req, -new, -batch, -nodes, -keyout, /tmp/tls2233839023174323260key, -out, /tmp/tls6216474238382410459csr, -subj, /O=io.strimzi/CN=my-cluster-entity-operator] 2020-04-14 23:08:05 DEBUG OpenSslCertManager:394 - Generating a 2048 bit RSA private key ..................+++ ...............................................+++ unable to write 'random state' writing new private key to '/tmp/tls2233839023174323260key' ----- 2020-04-14 23:08:05 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:08:05 DEBUG OpenSslCertManager:383 - Running command [openssl, x509, -req, -days, 365, -in, /tmp/tls6216474238382410459csr, -CA, /tmp/ca-crt-6763440867423886104.crt, -CAkey, /tmp/ca-key-5398215496423413423.key, -CAcreateserial, -out, /tmp/tls1417612639322098570cert] 2020-04-14 23:08:05 DEBUG OpenSslCertManager:394 - Signature ok subject=/O=io.strimzi/CN=my-cluster-entity-operator Getting CA Private Key unable to write 'random state' 2020-04-14 23:08:05 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:08:05 DEBUG OpenSslCertManager:383 - Running command [openssl, pkcs12, -export, -in, /tmp/tls1417612639322098570cert, -inkey, /tmp/tls2233839023174323260key, -name, my-cluster-entity-operator, -out, /tmp/tls7681306415567801064p12, -passout, pass:Vai9zOvlKwoL] 2020-04-14 23:08:05 DEBUG OpenSslCertManager:394 - unable to write 'random state' 2020-04-14 23:08:05 DEBUG OpenSslCertManager:395 - result 0 2020-04-14 23:08:05 DEBUG ModelUtils:252 - End generating certificates 2020-04-14 23:08:05 DEBUG SecretOperator:99 - Secret kafka-cluster-test/my-cluster-entity-operator-certs does not exist, creating it 2020-04-14 23:08:05 DEBUG SecretOperator:195 - Secret my-cluster-entity-operator-certs in namespace kafka-cluster-test has been created 2020-04-14 23:08:05 DEBUG DeploymentOperator:99 - Deployment kafka-cluster-test/my-cluster-entity-operator does not exist, creating it 2020-04-14 23:08:05 DEBUG DeploymentOperator:195 - Deployment my-cluster-entity-operator in namespace kafka-cluster-test has been created 2020-04-14 23:08:05 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:08:05 WARN AbstractOperator:246 - Reconciliation #51(timer) Kafka(kafka-cluster-test/my-cluster): Failed to acquire lock lock::kafka-cluster-test::Kafka::my-cluster within 10000ms. 2020-04-14 23:08:06 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 23:08:06 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:08:25 INFO ClusterOperator:127 - Triggering periodic reconciliation for namespace kafka-cluster-test... 2020-04-14 23:08:33 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 23:08:33 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:33 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-exporter-certs does not exist, noop 2020-04-14 23:08:33 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:33 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:33 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 23:08:33 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-jmxtrans-config does not exist, noop 2020-04-14 23:08:33 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 23:08:33 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/type","value":"Warning"} 2020-04-14 23:08:33 DEBUG StatusDiff:43 - Current Status path /conditions/0/type has value "NotReady" 2020-04-14 23:08:33 DEBUG StatusDiff:44 - Desired Status path /conditions/0/type has value "Warning" 2020-04-14 23:08:33 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2020-04-14T23:06:02+0000"} 2020-04-14 23:08:33 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/reason","value":"KafkaStorage"} 2020-04-14 23:08:33 DEBUG StatusDiff:43 - Current Status path /conditions/0/reason has value "Creating" 2020-04-14 23:08:33 DEBUG StatusDiff:44 - Desired Status path /conditions/0/reason has value "KafkaStorage" 2020-04-14 23:08:33 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/conditions/0/message","value":"A Kafka cluster with a single replica and ephemeral storage will lose topic messages after any restart or rolling update."} 2020-04-14 23:08:33 DEBUG StatusDiff:43 - Current Status path /conditions/0/message has value "Kafka cluster is being deployed" 2020-04-14 23:08:33 DEBUG StatusDiff:44 - Desired Status path /conditions/0/message has value "A Kafka cluster with a single replica and ephemeral storage will lose topic messages after any restart or rolling update." 2020-04-14 23:08:33 DEBUG StatusDiff:42 - Status differs: {"op":"add","path":"/conditions/1","value":{"type":"Warning","status":"True","lastTransitionTime":"2020-04-14T23:06:02+0000","reason":"ZooKeeperStorage","message":"A ZooKeeper cluster with a single replica and ephemeral storage will be in a defective state after any restart or rolling update. It is recommended that a minimum of three replicas are used."}} 2020-04-14 23:08:33 DEBUG StatusDiff:43 - Current Status path /conditions/1 has value 2020-04-14 23:08:33 DEBUG StatusDiff:44 - Desired Status path /conditions/1 has value {"type":"Warning","status":"True","lastTransitionTime":"2020-04-14T23:06:02+0000","reason":"ZooKeeperStorage","message":"A ZooKeeper cluster with a single replica and ephemeral storage will be in a defective state after any restart or rolling update. It is recommended that a minimum of three replicas are used."} 2020-04-14 23:08:33 DEBUG StatusDiff:42 - Status differs: {"op":"add","path":"/conditions/2","value":{"type":"Ready","status":"True","lastTransitionTime":"2020-04-14T23:08:33+0000"}} 2020-04-14 23:08:33 DEBUG StatusDiff:43 - Current Status path /conditions/2 has value 2020-04-14 23:08:33 DEBUG StatusDiff:44 - Desired Status path /conditions/2 has value {"type":"Ready","status":"True","lastTransitionTime":"2020-04-14T23:08:33+0000"} 2020-04-14 23:08:33 DEBUG StatusDiff:42 - Status differs: {"op":"replace","path":"/observedGeneration","value":1} 2020-04-14 23:08:33 DEBUG StatusDiff:43 - Current Status path /observedGeneration has value 0 2020-04-14 23:08:33 DEBUG StatusDiff:44 - Desired Status path /observedGeneration has value 1 2020-04-14 23:08:33 DEBUG StatusDiff:42 - Status differs: {"op":"add","path":"/listeners","value":[{"type":"plain","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9092}]},{"type":"tls","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9093}],"certificates":["-----BEGIN CERTIFICATE-----\nMIIDLTCCAhWgAwIBAgIJAN7a/ox+enxJMA0GCSqGSIb3DQEBCwUAMC0xEzARBgNV\nBAoMCmlvLnN0cmltemkxFjAUBgNVBAMMDWNsdXN0ZXItY2EgdjAwHhcNMjAwNDE0\nMjMwNTM5WhcNMjEwNDE0MjMwNTM5WjAtMRMwEQYDVQQKDAppby5zdHJpbXppMRYw\nFAYDVQQDDA1jbHVzdGVyLWNhIHYwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB\nCgKCAQEAs+9mXkYdNWRczpLqMh7J3Aungu3d9ZI2Rb+ZB1MVvi7jlPkGRSnvQkzX\nd/RThwkFtDX75+HvJvo4FQjAboZkAjNlYcNClkMJNlMgfcwFPZ+iKiLtff0zovrC\n/obroXhFGWah1lOcw0AKpIHUf52G7lGWllJE6b2Rrvp7ynevOcy2VN1q0xamqZW2\nDL7SHX7RO7HvKl8ugJm+0nI5uhrA0SSkxpq9DMQcJiJoCDzjA5CRaH3blv24lhij\nHAe45H3KHvq0lg+DfNQbnN+4ulOENPc9RX4jZV72iv13WodHx6z4ue4v7nUtqmQk\ndoUTx7+PNKCidWMkfNY6+2ywl379pwIDAQABo1AwTjAdBgNVHQ4EFgQUSkFC7Zir\nvJ8Xwq0EXMhhHkpKHeIwHwYDVR0jBBgwFoAUSkFC7ZirvJ8Xwq0EXMhhHkpKHeIw\nDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAfNOqxlXrhBFublug4MMH\nKskjHu241d6LjlanVAjZ2+YsPtgS+NsjZD7Tdgr3Duc7ab3FkS4V1Qovgghwiw/2\nG/HgTgJwfHVbLFseYm/WGKdc28vBJg/KjPI9cMYTyJ16KtCraia+GXv8mXOZN/WL\nKivnsHF5WPwYDmUNIJNgjqEb0Y4NKHJxWOrn4e4Abqj8HEKvVbeUrujCU2bzwP9j\nh132RkvaLExXvRgKMAS2LCDLCI2/RKXAMQk+fc1rBE37JHCYy3AULx10aLa+zfMg\n4H+E+wwvcv6z6vzJFnN3dMGq25qyfbKqJi+ZQTWqOrrKRm6jir3bPfzzVlCoKxrZ\nbA==\n-----END CERTIFICATE-----\n"]}]} 2020-04-14 23:08:33 DEBUG StatusDiff:43 - Current Status path /listeners has value 2020-04-14 23:08:33 DEBUG StatusDiff:44 - Desired Status path /listeners has value [{"type":"plain","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9092}]},{"type":"tls","addresses":[{"host":"my-cluster-kafka-bootstrap.kafka-cluster-test.svc","port":9093}],"certificates":["-----BEGIN CERTIFICATE-----\nMIIDLTCCAhWgAwIBAgIJAN7a/ox+enxJMA0GCSqGSIb3DQEBCwUAMC0xEzARBgNV\nBAoMCmlvLnN0cmltemkxFjAUBgNVBAMMDWNsdXN0ZXItY2EgdjAwHhcNMjAwNDE0\nMjMwNTM5WhcNMjEwNDE0MjMwNTM5WjAtMRMwEQYDVQQKDAppby5zdHJpbXppMRYw\nFAYDVQQDDA1jbHVzdGVyLWNhIHYwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB\nCgKCAQEAs+9mXkYdNWRczpLqMh7J3Aungu3d9ZI2Rb+ZB1MVvi7jlPkGRSnvQkzX\nd/RThwkFtDX75+HvJvo4FQjAboZkAjNlYcNClkMJNlMgfcwFPZ+iKiLtff0zovrC\n/obroXhFGWah1lOcw0AKpIHUf52G7lGWllJE6b2Rrvp7ynevOcy2VN1q0xamqZW2\nDL7SHX7RO7HvKl8ugJm+0nI5uhrA0SSkxpq9DMQcJiJoCDzjA5CRaH3blv24lhij\nHAe45H3KHvq0lg+DfNQbnN+4ulOENPc9RX4jZV72iv13WodHx6z4ue4v7nUtqmQk\ndoUTx7+PNKCidWMkfNY6+2ywl379pwIDAQABo1AwTjAdBgNVHQ4EFgQUSkFC7Zir\nvJ8Xwq0EXMhhHkpKHeIwHwYDVR0jBBgwFoAUSkFC7ZirvJ8Xwq0EXMhhHkpKHeIw\nDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAfNOqxlXrhBFublug4MMH\nKskjHu241d6LjlanVAjZ2+YsPtgS+NsjZD7Tdgr3Duc7ab3FkS4V1Qovgghwiw/2\nG/HgTgJwfHVbLFseYm/WGKdc28vBJg/KjPI9cMYTyJ16KtCraia+GXv8mXOZN/WL\nKivnsHF5WPwYDmUNIJNgjqEb0Y4NKHJxWOrn4e4Abqj8HEKvVbeUrujCU2bzwP9j\nh132RkvaLExXvRgKMAS2LCDLCI2/RKXAMQk+fc1rBE37JHCYy3AULx10aLa+zfMg\n4H+E+wwvcv6z6vzJFnN3dMGq25qyfbKqJi+ZQTWqOrrKRm6jir3bPfzzVlCoKxrZ\nbA==\n-----END CERTIFICATE-----\n"]}] 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:466 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Completed status update 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:229 - Status for my-cluster is up to date 2020-04-14 23:08:33 INFO AbstractOperator:317 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): reconciled 2020-04-14 23:08:33 DEBUG AbstractOperator:237 - Reconciliation #45(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 23:08:33 DEBUG AbstractOperator:226 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 23:08:33 INFO OperatorWatcher:40 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster in namespace kafka-cluster-test was MODIFIED 2020-04-14 23:08:33 INFO AbstractOperator:173 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:520 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Status is already set. No need to set initial status 2020-04-14 23:08:33 DEBUG Ca:653 - cluster-ca: The CA certificate in secret my-cluster-cluster-ca-cert already exists and does not need renewing 2020-04-14 23:08:33 DEBUG Ca:512 - cluster-ca renewalType NOOP 2020-04-14 23:08:33 DEBUG Ca:653 - clients-ca: The CA certificate in secret my-cluster-clients-ca-cert already exists and does not need renewing 2020-04-14 23:08:33 DEBUG Ca:512 - clients-ca renewalType NOOP 2020-04-14 23:08:33 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca-cert already exists, patching it 2020-04-14 23:08:33 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert already exists, patching it 2020-04-14 23:08:33 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca already exists, patching it 2020-04-14 23:08:33 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca already exists, patching it 2020-04-14 23:08:33 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs already exists, patching it 2020-04-14 23:08:33 DEBUG SecretOperator:168 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 WARN AbstractModel:214 - A ZooKeeper cluster with a single replica and ephemeral storage will be in a defective state after any restart or rolling update. It is recommended that a minimum of three replicas are used. 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-zookeeper 2020-04-14 23:08:33 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper already exists, patching it 2020-04-14 23:08:33 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 23:08:33 DEBUG KafkaAssemblyOperator:783 - Kafka.spec.kafka.version is unchanged therefore no change to Zookeeper is required 2020-04-14 23:08:33 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:08:33 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-client already exists, patching it 2020-04-14 23:08:33 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 23:08:33 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config already exists, patching it 2020-04-14 23:08:33 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:33 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 23:08:33 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 23:08:33 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 23:08:33 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-0 already exists 2020-04-14 23:08:33 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 23:08:33 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 23:08:33 DEBUG SecretOperator:168 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 23:08:33 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:08:33 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been deleted 2020-04-14 23:08:33 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 23:08:33 DEBUG ZookeeperSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:93 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"replace","path":"/spec/template/spec/containers/0/resources/limits/cpu","value":"0.5"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:93 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"replace","path":"/spec/template/spec/containers/0/resources/requests/memory","value":"0.5G"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/1/configMap/defaultMode"} 2020-04-14 23:08:33 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/status"} 2020-04-14 23:08:33 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-zookeeper 2020-04-14 23:08:33 DEBUG ZookeeperSetOperator:168 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 23:08:34 DEBUG ZookeeperSetOperator:74 - Considering rolling update of kafka-cluster-test/my-cluster-zookeeper 2020-04-14 23:08:34 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:08:34 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:08:34 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:08:34 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-0. 2020-04-14 23:08:34 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:34 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:34 DEBUG KafkaAssemblyOperator:1538 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Verifying that Zookeeper is configured to run with 1 replicas 2020-04-14 23:08:34 DEBUG ZookeeperScaler:62 - Creating Zookeeper Scaler for cluster my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:08:34 DEBUG Util:62 - Waiting for ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 to get ready 2020-04-14 23:08:34 DEBUG SslHandler:1743 - [id: 0x70c0b301, L:/172.17.0.6:48424 - R:my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc/172.17.0.9:2181] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 2020-04-14 23:08:34 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:SyncConnected type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:08:35 DEBUG Util:84 - ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 is ready 2020-04-14 23:08:35 DEBUG ZookeeperScaler:198 - Current Zookeeper configuration is {server.1=127.0.0.1:28880:38880:participant;127.0.0.1:21810} 2020-04-14 23:08:35 DEBUG ZookeeperScaler:181 - The Zookeeper server configuration is already up to date 2020-04-14 23:08:35 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:Closed type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-kafka 2020-04-14 23:08:35 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-kafka already exists, patching it 2020-04-14 23:08:35 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-kafka in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:920 - Kafka.spec.kafka.version unchanged 2020-04-14 23:08:35 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:08:35 DEBUG ClusterRoleBindingOperator:110 - ClusterRoleBinding strimzi-kafka-cluster-test-my-cluster-kafka-init does not exist, noop 2020-04-14 23:08:35 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-bootstrap already exists, patching it 2020-04-14 23:08:35 DEBUG ServiceOperator:168 - Service my-cluster-kafka-bootstrap in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 23:08:35 DEBUG ServiceOperator:168 - Service my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-external-bootstrap does not exist, noop 2020-04-14 23:08:35 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:08:35 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, noop 2020-04-14 23:08:35 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:08:35 DEBUG AbstractModel:739 - Generating certificates 2020-04-14 23:08:35 DEBUG Ca:193 - cluster-ca: Reconciling kafka broker certificates 2020-04-14 23:08:35 DEBUG Ca:361 - Certificate for my-cluster-kafka-0 already exists 2020-04-14 23:08:35 DEBUG AbstractModel:747 - End generating certificates 2020-04-14 23:08:35 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-kafka-config already exists, patching it 2020-04-14 23:08:35 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-kafka-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-jmx does not exist, noop 2020-04-14 23:08:35 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:08:35 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been deleted 2020-04-14 23:08:35 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 23:08:35 DEBUG KafkaSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:93 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"replace","path":"/spec/template/spec/containers/0/resources/limits/memory","value":"1.5Gi"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/4/configMap/defaultMode"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/status"} 2020-04-14 23:08:35 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-kafka 2020-04-14 23:08:35 DEBUG KafkaSetOperator:168 - StatefulSet my-cluster-kafka in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG KafkaRoller:150 - Initial order for rolling restart [0] 2020-04-14 23:08:35 DEBUG KafkaRoller:194 - Considering restart of pod 0 after delay of 0 MILLISECONDS 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:08:35 DEBUG KafkaRoller:281 - Pod 0 does not need to be restarted 2020-04-14 23:08:35 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 0 to become ready 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG KafkaRoller:284 - Pod 0 is now ready 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-0. 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:35 DEBUG RoleBindingOperator:111 - RoleBinding kafka-cluster-test/strimzi-my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:35 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-topic-operator-config does not exist, noop 2020-04-14 23:08:35 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-topic-operator-certs does not exist, noop 2020-04-14 23:08:35 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:35 DEBUG AbstractModel:361 - logging is not set, using default loggers 2020-04-14 23:08:35 DEBUG AbstractModel:361 - logging is not set, using default loggers 2020-04-14 23:08:35 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-entity-operator already exists, patching it 2020-04-14 23:08:35 DEBUG RoleBindingOperator:102 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-topic-operator already exists, patching it 2020-04-14 23:08:35 DEBUG RoleBindingOperator:168 - RoleBinding strimzi-my-cluster-entity-topic-operator in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG RoleBindingOperator:102 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-user-operator already exists, patching it 2020-04-14 23:08:35 DEBUG RoleBindingOperator:168 - RoleBinding strimzi-my-cluster-entity-user-operator in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-entity-topic-operator-config already exists, patching it 2020-04-14 23:08:35 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-entity-topic-operator-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:35 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-entity-user-operator-config already exists, patching it 2020-04-14 23:08:35 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-entity-user-operator-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-entity-operator-certs already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-entity-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG DeploymentOperator:102 - Deployment kafka-cluster-test/my-cluster-entity-operator already exists, patching it 2020-04-14 23:08:35 DEBUG DeploymentOperator:168 - Deployment my-cluster-entity-operator in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:35 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-exporter-certs does not exist, noop 2020-04-14 23:08:35 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:35 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:35 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 23:08:35 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-jmxtrans-config does not exist, noop 2020-04-14 23:08:35 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 23:08:35 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2020-04-14T23:08:35+0000"} 2020-04-14 23:08:35 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/1/lastTransitionTime","value":"2020-04-14T23:08:35+0000"} 2020-04-14 23:08:35 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/2/lastTransitionTime","value":"2020-04-14T23:08:35+0000"} 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:474 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Status did not change 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:229 - Status for my-cluster is up to date 2020-04-14 23:08:35 INFO AbstractOperator:317 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): reconciled 2020-04-14 23:08:35 DEBUG AbstractOperator:237 - Reconciliation #52(timer) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 23:08:35 DEBUG AbstractOperator:226 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster acquired 2020-04-14 23:08:35 INFO AbstractOperator:173 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Kafka my-cluster should be created or updated 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:520 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Status is already set. No need to set initial status 2020-04-14 23:08:35 DEBUG Ca:653 - cluster-ca: The CA certificate in secret my-cluster-cluster-ca-cert already exists and does not need renewing 2020-04-14 23:08:35 DEBUG Ca:512 - cluster-ca renewalType NOOP 2020-04-14 23:08:35 DEBUG Ca:653 - clients-ca: The CA certificate in secret my-cluster-clients-ca-cert already exists and does not need renewing 2020-04-14 23:08:35 DEBUG Ca:512 - clients-ca renewalType NOOP 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca-cert already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca-cert already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca-cert in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-clients-ca already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-ca already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-clients-ca in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-cluster-ca in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-cluster-operator-certs already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-cluster-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 WARN AbstractModel:214 - A ZooKeeper cluster with a single replica and ephemeral storage will be in a defective state after any restart or rolling update. It is recommended that a minimum of three replicas are used. 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-zookeeper 2020-04-14 23:08:35 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-zookeeper already exists, patching it 2020-04-14 23:08:35 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:783 - Kafka.spec.kafka.version is unchanged therefore no change to Zookeeper is required 2020-04-14 23:08:35 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:08:35 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-client already exists, patching it 2020-04-14 23:08:35 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-client in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 23:08:35 DEBUG ServiceOperator:168 - Service my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-zookeeper-config already exists, patching it 2020-04-14 23:08:35 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-zookeeper-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:35 DEBUG AbstractModel:513 - Generating certificates 2020-04-14 23:08:35 DEBUG AbstractModel:516 - Cluster communication certificates 2020-04-14 23:08:35 DEBUG Ca:138 - cluster-ca: Reconciling zookeeper certificates 2020-04-14 23:08:35 DEBUG Ca:361 - Certificate for my-cluster-zookeeper-0 already exists 2020-04-14 23:08:35 DEBUG AbstractModel:518 - End generating certificates 2020-04-14 23:08:35 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-zookeeper-nodes already exists, patching it 2020-04-14 23:08:35 DEBUG SecretOperator:168 - Secret my-cluster-zookeeper-nodes in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:08:35 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been deleted 2020-04-14 23:08:35 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-zookeeper in namespace kafka-cluster-test has been created 2020-04-14 23:08:35 DEBUG ZookeeperSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-zookeeper already exists, patching it 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:93 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"replace","path":"/spec/template/spec/containers/0/resources/limits/cpu","value":"0.5"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:93 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"replace","path":"/spec/template/spec/containers/0/resources/requests/memory","value":"0.5G"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/1/configMap/defaultMode"} 2020-04-14 23:08:35 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-zookeeper ignoring diff {"op":"remove","path":"/status"} 2020-04-14 23:08:35 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-zookeeper 2020-04-14 23:08:35 DEBUG ZookeeperSetOperator:168 - StatefulSet my-cluster-zookeeper in namespace kafka-cluster-test has been patched 2020-04-14 23:08:35 DEBUG ZookeeperSetOperator:74 - Considering rolling update of kafka-cluster-test/my-cluster-zookeeper 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-zookeeper: pod my-cluster-zookeeper-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-zookeeper-0. 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:35 DEBUG Util:84 - Pods resource my-cluster-zookeeper-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:35 DEBUG KafkaAssemblyOperator:1538 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Verifying that Zookeeper is configured to run with 1 replicas 2020-04-14 23:08:35 DEBUG ZookeeperScaler:62 - Creating Zookeeper Scaler for cluster my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:08:35 DEBUG Util:62 - Waiting for ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 to get ready 2020-04-14 23:08:35 DEBUG SslHandler:1743 - [id: 0xce8f4cce, L:/172.17.0.6:48458 - R:my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc/172.17.0.9:2181] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 2020-04-14 23:08:35 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:SyncConnected type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:08:36 DEBUG Util:84 - ZooKeeperAdmin connection to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 is ready 2020-04-14 23:08:36 DEBUG ZookeeperScaler:198 - Current Zookeeper configuration is {server.1=127.0.0.1:28880:38880:participant;127.0.0.1:21810} 2020-04-14 23:08:36 DEBUG ZookeeperScaler:181 - The Zookeeper server configuration is already up to date 2020-04-14 23:08:36 DEBUG ZookeeperScaler:139 - Received event WatchedEvent state:Closed type:None path:null from ZooKeeperAdmin client connected to my-cluster-zookeeper-0.my-cluster-zookeeper-nodes.kafka-cluster-test.svc:2181 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test to get ready 2020-04-14 23:08:36 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-client in namespace kafka-cluster-test is ready 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test to get ready 2020-04-14 23:08:36 DEBUG Util:84 - Endpoints resource my-cluster-zookeeper-nodes in namespace kafka-cluster-test is ready 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:2562 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Considering manual cleaning of Pods for StatefulSet my-cluster-kafka 2020-04-14 23:08:36 DEBUG NetworkPolicyOperator:102 - NetworkPolicy kafka-cluster-test/my-cluster-network-policy-kafka already exists, patching it 2020-04-14 23:08:36 DEBUG NetworkPolicyOperator:168 - NetworkPolicy my-cluster-network-policy-kafka in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:850 - STS my-cluster-kafka currently has Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:860 - STS my-cluster-kafka is moving from Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:870 - STS my-cluster-kafka is moving to Kafka version KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:874 - Kafka version change: Kafka version=KafkaVersion{version='2.4.0', protocolVersion='2.4', messageVersion='2.4', zookeeperVersion='3.5.6', isDefault=true, unsupportedFeatures='null'} (no version change) 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:920 - Kafka.spec.kafka.version unchanged 2020-04-14 23:08:36 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:08:36 DEBUG ClusterRoleBindingOperator:110 - ClusterRoleBinding strimzi-kafka-cluster-test-my-cluster-kafka-init does not exist, noop 2020-04-14 23:08:36 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-bootstrap already exists, patching it 2020-04-14 23:08:36 DEBUG ServiceOperator:168 - Service my-cluster-kafka-bootstrap in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG ServiceOperator:102 - Service kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 23:08:36 DEBUG ServiceOperator:168 - Service my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-external-bootstrap does not exist, noop 2020-04-14 23:08:36 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:08:36 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-bootstrap does not exist, noop 2020-04-14 23:08:36 DEBUG RouteOperator:111 - Route kafka-cluster-test/my-cluster-kafka-0 does not exist, noop 2020-04-14 23:08:36 DEBUG AbstractModel:739 - Generating certificates 2020-04-14 23:08:36 DEBUG Ca:193 - cluster-ca: Reconciling kafka broker certificates 2020-04-14 23:08:36 DEBUG Ca:361 - Certificate for my-cluster-kafka-0 already exists 2020-04-14 23:08:36 DEBUG AbstractModel:747 - End generating certificates 2020-04-14 23:08:36 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-kafka-config already exists, patching it 2020-04-14 23:08:36 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-kafka-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:36 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-kafka-brokers already exists, patching it 2020-04-14 23:08:36 DEBUG SecretOperator:168 - Secret my-cluster-kafka-brokers in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-jmx does not exist, noop 2020-04-14 23:08:36 DEBUG PodDisruptionBudgetOperator:102 - PodDisruptionBudget kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:08:36 DEBUG PodDisruptionBudgetOperator:149 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been deleted 2020-04-14 23:08:36 DEBUG PodDisruptionBudgetOperator:195 - PodDisruptionBudget my-cluster-kafka in namespace kafka-cluster-test has been created 2020-04-14 23:08:36 DEBUG KafkaSetOperator:102 - StatefulSet kafka-cluster-test/my-cluster-kafka already exists, patching it 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/revisionHistoryLimit"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/metadata/annotations/strimzi.io~1generation"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/failureThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/periodSeconds"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/livenessProbe/successThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/failureThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/periodSeconds"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/readinessProbe/successThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:93 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"replace","path":"/spec/template/spec/containers/0/resources/limits/memory","value":"1.5Gi"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePath"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/0/terminationMessagePolicy"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/failureThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/periodSeconds"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/livenessProbe/successThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/failureThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/periodSeconds"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/readinessProbe/successThreshold"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/resources"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePath"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/containers/1/terminationMessagePolicy"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/dnsPolicy"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/restartPolicy"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/securityContext"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/serviceAccount"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/spec/template/spec/volumes/4/configMap/defaultMode"} 2020-04-14 23:08:36 DEBUG StatefulSetDiff:84 - StatefulSet kafka-cluster-test/my-cluster-kafka ignoring diff {"op":"remove","path":"/status"} 2020-04-14 23:08:36 DEBUG StatefulSetOperator:294 - Patching StatefulSet kafka-cluster-test/my-cluster-kafka 2020-04-14 23:08:36 DEBUG KafkaSetOperator:168 - StatefulSet my-cluster-kafka in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG KafkaRoller:150 - Initial order for rolling restart [0] 2020-04-14 23:08:36 DEBUG KafkaRoller:194 - Considering restart of pod 0 after delay of 0 MILLISECONDS 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:3046 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/generation=0; sts has strimzi.io/generation=0 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-tls-listener-thumbprint=; sts has strimzi.io/custom-cert-tls-listener-thumbprint= 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:3064 - Rolling update of kafka-cluster-test/my-cluster-kafka: pod my-cluster-kafka-0 has strimzi.io/custom-cert-external-listener-thumbprint=; sts has strimzi.io/custom-cert-external-listener-thumbprint= 2020-04-14 23:08:36 DEBUG KafkaRoller:281 - Pod 0 does not need to be restarted 2020-04-14 23:08:36 DEBUG KafkaRoller:282 - Waiting for non-restarted pod 0 to become ready 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:36 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:36 DEBUG KafkaRoller:284 - Pod 0 is now ready 2020-04-14 23:08:36 DEBUG KafkaAssemblyOperator:2474 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Checking readiness of pod my-cluster-kafka-0. 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test to get ready 2020-04-14 23:08:36 DEBUG Util:84 - Pods resource my-cluster-kafka-0 in namespace kafka-cluster-test is ready 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test to get ready 2020-04-14 23:08:36 DEBUG Util:84 - Endpoints resource my-cluster-kafka-bootstrap in namespace kafka-cluster-test is ready 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test to get ready 2020-04-14 23:08:36 DEBUG Util:84 - Endpoints resource my-cluster-kafka-brokers in namespace kafka-cluster-test is ready 2020-04-14 23:08:36 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:36 DEBUG RoleBindingOperator:111 - RoleBinding kafka-cluster-test/strimzi-my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:36 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-topic-operator-config does not exist, noop 2020-04-14 23:08:36 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-topic-operator-certs does not exist, noop 2020-04-14 23:08:36 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-topic-operator does not exist, noop 2020-04-14 23:08:36 DEBUG AbstractModel:361 - logging is not set, using default loggers 2020-04-14 23:08:36 DEBUG AbstractModel:361 - logging is not set, using default loggers 2020-04-14 23:08:36 DEBUG ServiceAccountOperator:102 - ServiceAccount kafka-cluster-test/my-cluster-entity-operator already exists, patching it 2020-04-14 23:08:36 DEBUG RoleBindingOperator:102 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-topic-operator already exists, patching it 2020-04-14 23:08:36 DEBUG RoleBindingOperator:168 - RoleBinding strimzi-my-cluster-entity-topic-operator in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG RoleBindingOperator:102 - RoleBinding kafka-cluster-test/strimzi-my-cluster-entity-user-operator already exists, patching it 2020-04-14 23:08:36 DEBUG RoleBindingOperator:168 - RoleBinding strimzi-my-cluster-entity-user-operator in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-entity-topic-operator-config already exists, patching it 2020-04-14 23:08:36 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-entity-topic-operator-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:36 DEBUG ConfigMapOperator:102 - ConfigMap kafka-cluster-test/my-cluster-entity-user-operator-config already exists, patching it 2020-04-14 23:08:36 DEBUG ConfigMapOperator:47 - ConfigMap my-cluster-entity-user-operator-config in namespace kafka-cluster-test has not been patched because resources are equal 2020-04-14 23:08:36 DEBUG SecretOperator:102 - Secret kafka-cluster-test/my-cluster-entity-operator-certs already exists, patching it 2020-04-14 23:08:36 DEBUG SecretOperator:168 - Secret my-cluster-entity-operator-certs in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG DeploymentOperator:102 - Deployment kafka-cluster-test/my-cluster-entity-operator already exists, patching it 2020-04-14 23:08:36 DEBUG DeploymentOperator:168 - Deployment my-cluster-entity-operator in namespace kafka-cluster-test has been patched 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:08:36 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 23:08:36 DEBUG Util:62 - Waiting for Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test to get ready 2020-04-14 23:08:37 DEBUG Util:84 - Deployment resource my-cluster-entity-operator in namespace kafka-cluster-test is ready 2020-04-14 23:08:37 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:37 DEBUG SecretOperator:111 - Secret kafka-cluster-test/my-cluster-kafka-exporter-certs does not exist, noop 2020-04-14 23:08:37 DEBUG ServiceOperator:111 - Service kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:37 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-exporter does not exist, noop 2020-04-14 23:08:37 DEBUG ServiceAccountOperator:111 - ServiceAccount kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 23:08:37 DEBUG ConfigMapOperator:111 - ConfigMap kafka-cluster-test/my-cluster-jmxtrans-config does not exist, noop 2020-04-14 23:08:37 DEBUG DeploymentOperator:111 - Deployment kafka-cluster-test/my-cluster-kafka-jmx-trans does not exist, noop 2020-04-14 23:08:37 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2020-04-14T23:08:36+0000"} 2020-04-14 23:08:37 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/1/lastTransitionTime","value":"2020-04-14T23:08:36+0000"} 2020-04-14 23:08:37 DEBUG StatusDiff:37 - Ignoring Status diff {"op":"replace","path":"/conditions/2/lastTransitionTime","value":"2020-04-14T23:08:37+0000"} 2020-04-14 23:08:37 DEBUG KafkaAssemblyOperator:474 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Status did not change 2020-04-14 23:08:37 DEBUG KafkaAssemblyOperator:229 - Status for my-cluster is up to date 2020-04-14 23:08:37 INFO AbstractOperator:317 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): reconciled 2020-04-14 23:08:37 DEBUG AbstractOperator:237 - Reconciliation #53(watch) Kafka(kafka-cluster-test/my-cluster): Lock lock::kafka-cluster-test::Kafka::my-cluster released 2020-04-14 23:08:37 DEBUG PoolThreadCache:250 - Freed 16 thread-local buffer(s) from thread: epollEventLoopGroup-14-1