This question already has answers here:
How to install Kafka on Windows?
(9 answers)
Closed 1 year ago.
I downloaded the kafka_2.13-3.0.0 in Window 10. I opened up CMD as admin. I was able to run zookeeper using the command bin\windows\zookeeper-server-start.bat config\zookeeper.properties. However, when i tried to start the kafka server with the following command
bin\windows\kafka-server-start.bat config\server.properties
, it gave me the following error" below. I am confused here because I can see the kafka-logs being created (see image below). Can someone help me out please ?
Output from running Zookeeper command
[2021-12-01 20:16:17,208] INFO Server environment:java.io.tmpdir=C:\Users\user123456~1\AppData\Local\Temp\ (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,208] INFO Server environment:java.compiler=<NA> (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,208] INFO Server environment:os.name=Windows 10 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,209] INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,209] INFO Server environment:os.version=10.0 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,209] INFO Server environment:user.name=user123456 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,210] INFO Server environment:user.home=C:\Users\user123456 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,210] INFO Server environment:user.dir=C:\kafka_2.13-3.0.0 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,211] INFO Server environment:os.memory.free=493MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,211] INFO Server environment:os.memory.max=512MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,211] INFO Server environment:os.memory.total=512MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,212] INFO zookeeper.enableEagerACLCheck = false (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,212] INFO zookeeper.digest.enabled = true (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,212] INFO zookeeper.closeSessionTxn.enabled = true (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,213] INFO zookeeper.flushDelay=0 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,213] INFO zookeeper.maxWriteQueuePollTime=0 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,214] INFO zookeeper.maxBatchSize=1000 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,214] INFO zookeeper.intBufferStartingSizeBytes = 1024 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,215] INFO Weighed connection throttling is disabled (org.apache.zookeeper.server.BlueThrottle)
[2021-12-01 20:16:17,216] INFO minSessionTimeout set to 6000 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,216] INFO maxSessionTimeout set to 60000 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,217] INFO Response cache size is initialized with value 400. (org.apache.zookeeper.server.ResponseCache)
[2021-12-01 20:16:17,217] INFO Response cache size is initialized with value 400. (org.apache.zookeeper.server.ResponseCache)
[2021-12-01 20:16:17,218] INFO zookeeper.pathStats.slotCapacity = 60 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2021-12-01 20:16:17,220] INFO zookeeper.pathStats.slotDuration = 15 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2021-12-01 20:16:17,221] INFO zookeeper.pathStats.maxDepth = 6 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2021-12-01 20:16:17,221] INFO zookeeper.pathStats.initialDelay = 5 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2021-12-01 20:16:17,221] INFO zookeeper.pathStats.delay = 5 (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2021-12-01 20:16:17,222] INFO zookeeper.pathStats.enabled = false (org.apache.zookeeper.server.util.RequestPathMetricsCollector)
[2021-12-01 20:16:17,224] INFO The max bytes for all large requests are set to 104857600 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,224] INFO The large request threshold is set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,224] INFO Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 60000 clientPortListenBacklog -1 datadir \tmp\zookeeper\version-2 snapdir \tmp\zookeeper\version-2 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,234] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2021-12-01 20:16:17,235] WARN maxCnxns is not configured, using default value 0. (org.apache.zookeeper.server.ServerCnxnFactory)
[2021-12-01 20:16:17,236] INFO Configuring NIO connection handler with 10s sessionless connection timeout, 2 selector thread(s), 32 worker threads, and 64 kB direct buffers. (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-12-01 20:16:17,238] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-12-01 20:16:17,250] INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory)
[2021-12-01 20:16:17,250] INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory)
[2021-12-01 20:16:17,251] INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase)
[2021-12-01 20:16:17,251] INFO zookeeper.commitLogCount=500 (org.apache.zookeeper.server.ZKDatabase)
[2021-12-01 20:16:17,256] INFO zookeeper.snapshot.compression.method = CHECKED (org.apache.zookeeper.server.persistence.SnapStream)
[2021-12-01 20:16:17,256] INFO Snapshotting: 0x0 to \tmp\zookeeper\version-2\snapshot.0 (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
[2021-12-01 20:16:17,260] INFO Snapshot loaded in 8 ms, highest zxid is 0x0, digest is 1371985504 (org.apache.zookeeper.server.ZKDatabase)
[2021-12-01 20:16:17,260] INFO Snapshotting: 0x0 to \tmp\zookeeper\version-2\snapshot.0 (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
[2021-12-01 20:16:17,261] INFO Snapshot taken in 1 ms (org.apache.zookeeper.server.ZooKeeperServer)
[2021-12-01 20:16:17,274] INFO PrepRequestProcessor (sid:0) started, reconfigEnabled=false (org.apache.zookeeper.server.PrepRequestProcessor)
[2021-12-01 20:16:17,274] INFO zookeeper.request_throttler.shutdownTimeout = 10000 (org.apache.zookeeper.server.RequestThrottler)
[2021-12-01 20:16:17,286] INFO Using checkIntervalMs=60000 maxPerMinute=10000 maxNeverUsedIntervalMs=0 (org.apache.zookeeper.server.ContainerManager)
[2021-12-01 20:16:17,287] INFO ZooKeeper audit is disabled. (org.apache.zookeeper.audit.ZKAuditProvider)
Output from running kafka command
[2021-12-01 20:18:22,577] INFO Client environment:java.io.tmpdir=C:\Users\user123456~1\AppData\Local\Temp\ (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,577] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,578] INFO Client environment:os.name=Windows 10 (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,579] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,579] INFO Client environment:os.version=10.0 (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,580] INFO Client environment:user.name=user123456 (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,581] INFO Client environment:user.home=C:\Users\user123456 (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,581] INFO Client environment:user.dir=C:\kafka_2.13-3.0.0 (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,582] INFO Client environment:os.memory.free=1011MB (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,583] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,583] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,585] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$#74f0ea28 (org.apache.zookeeper.ZooKeeper)
[2021-12-01 20:18:22,593] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2021-12-01 20:18:22,597] INFO zookeeper.request.timeout value is 0. feature enabled=false (org.apache.zookeeper.ClientCnxn)
[2021-12-01 20:18:22,598] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2021-12-01 20:18:22,601] INFO Opening socket connection to server localhost/127.0.0.1:2181. (org.apache.zookeeper.ClientCnxn)
[2021-12-01 20:18:22,601] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2021-12-01 20:18:22,602] INFO Socket connection established, initiating session, client: /127.0.0.1:57191, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2021-12-01 20:18:22,617] INFO Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x1000008c9c20000, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)
[2021-12-01 20:18:22,619] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)
[2021-12-01 20:18:22,720] INFO [feature-zk-node-event-process-thread]: Starting (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2021-12-01 20:18:22,729] INFO Feature ZK node at path: /feature does not exist (kafka.server.FinalizedFeatureChangeListener)
[2021-12-01 20:18:22,729] INFO Cleared cache (kafka.server.FinalizedFeatureCache)
[2021-12-01 20:18:22,844] INFO Cluster ID = j7KyANF-S2OuLlr_IYyqDw (kafka.server.KafkaServer)
[2021-12-01 20:18:22,846] WARN No meta.properties file under dir C:\tmp\kafka-logs\meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2021-12-01 20:18:22,876] INFO KafkaConfig values:
...
[2021-12-01 20:18:22,918] INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2021-12-01 20:18:22,919] INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2021-12-01 20:18:22,919] INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2021-12-01 20:18:22,920] INFO [ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2021-12-01 20:18:22,931] INFO Log directory C:\tmp\kafka-logs not found, creating it. (kafka.log.LogManager)
[2021-12-01 20:18:22,933] ERROR Failed to create or validate data directory C:\tmp\kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: C:\tmp
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:335)
at org.apache.kafka.common.utils.Utils.flushDir(Utils.java:953)
at kafka.log.LogManager.$anonfun$createAndValidateLogDirs$1(LogManager.scala:159)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
at kafka.log.LogManager.createAndValidateLogDirs(LogManager.scala:149)
at kafka.log.LogManager.<init>(LogManager.scala:88)
at kafka.log.LogManager$.apply(LogManager.scala:1283)
at kafka.server.KafkaServer.startup(KafkaServer.scala:254)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2021-12-01 20:18:22,936] ERROR Shutdown broker because none of the specified log dirs from C:\tmp\kafka-logs can be created or validated (kafka.log.LogManager)
I also tried running icacls "C:\tmp\" /q /c /t /grant Users:F. I am still getting the same error.
It turns out that 3.0.0 has issues with Windows. I downgraded to 2.8.1 and things worked fine. I hate Windows.
Related
I'm using Confluent Community 6.0.1. Three nodes Kafka cluster:
devKafka04: Kafka Broker1, Zookeeper 1
devKafka05: Kafka Broker2, Zookeeper 2
devKafka06: Kafka Broker3, Zookeeper 3
The SSL encryption is already working well on the Kafka Brokers.
I'd like to add SASL to enable mutual authentication between Kafka and Zookeeper.
I was following the Confluent document:
https://docs.confluent.io/platform/current/kafka/incremental-security-upgrade.html#adding-security-to-a-running-zk-cluster
[Updates] After I applied the changes, Zookeeper could not start on the secureclientPort. That's why the Kafka broker couldn't start. Here are the error log and docker compose configurations.
I'm wondering if there's something with the confluent zookeeper image.
Please help me out. Thanks.
$ sudo docker logs zookeeper
===> User
uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
===> Configuring ...
===> Running preflight checks ...
===> Check if /var/lib/zookeeper/data is writable ...
===> Check if /var/lib/zookeeper/log is writable ...
===> Launching ...
===> Printing /var/lib/zookeeper/data/myid
1===> Launching zookeeper ...
[2021-03-24 19:03:08,857] INFO Reading configuration from: /etc/kafka/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2021-03-24 19:03:08,862] INFO clientPortAddress is 0.0.0.0:2181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2021-03-24 19:03:08,862] INFO secureClientPort is not set (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2021-03-24 19:03:08,876] INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager)
[2021-03-24 19:03:08,876] INFO autopurge.purgeInterval set to 0 (org.apache.zookeeper.server.DatadirCleanupManager)
[2021-03-24 19:03:08,876] INFO Purge task is not scheduled. (org.apache.zookeeper.server.DatadirCleanupManager)
[2021-03-24 19:03:08,880] INFO Log4j 1.2 jmx support found and enabled. (org.apache.zookeeper.jmx.ManagedUtil)
[2021-03-24 19:03:08,904] INFO Starting quorum peer (org.apache.zookeeper.server.quorum.QuorumPeerMain)
[2021-03-24 19:03:08,909] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2021-03-24 19:03:08,917] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2021-03-24 19:03:08,953] INFO Server successfully logged in. (org.apache.zookeeper.Login)
[2021-03-24 19:03:08,957] INFO Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers. (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-03-24 19:03:08,961] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-03-24 19:03:08,986] INFO Logging initialized #929ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log)
[2021-03-24 19:03:09,081] WARN o.e.j.s.ServletContextHandler#6c2c1385{/,null,UNAVAILABLE} contextPath ends with /* (org.eclipse.jetty.server.handler.ContextHandler)
[2021-03-24 19:03:09,082] WARN Empty contextPath (org.eclipse.jetty.server.handler.ContextHandler)
[2021-03-24 19:03:09,097] INFO zookeeper.snapshot.trust.empty : false (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
[2021-03-24 19:03:09,102] INFO Local sessions disabled (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,102] INFO Local session upgrading disabled (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,102] INFO tickTime set to 3000 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,102] INFO minSessionTimeout set to 6000 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,102] INFO maxSessionTimeout set to 60000 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,102] INFO initLimit set to 10 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,115] INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase)
[2021-03-24 19:03:09,116] INFO Using insecure (non-TLS) quorum communication (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,117] INFO Port unification disabled (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,117] INFO QuorumPeer communication is not secured! (SASL auth disabled) (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,117] INFO quorum.cnxn.threads.size set to 20 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,118] INFO Reading snapshot /var/lib/zookeeper/data/version-2/snapshot.a00000000 (org.apache.zookeeper.server.persistence.FileSnap)
[2021-03-24 19:03:09,213] INFO jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.9.1+1-LTS (org.eclipse.jetty.server.Server)
[2021-03-24 19:03:09,261] INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session)
[2021-03-24 19:03:09,261] INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session)
[2021-03-24 19:03:09,263] INFO node0 Scavenging every 660000ms (org.eclipse.jetty.server.session)
[2021-03-24 19:03:09,272] INFO Started o.e.j.s.ServletContextHandler#6c2c1385{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)
[2021-03-24 19:03:09,281] INFO Started ServerConnector#6d07a63d{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.server.AbstractConnector)
[2021-03-24 19:03:09,281] INFO Started #1224ms (org.eclipse.jetty.server.Server)
[2021-03-24 19:03:09,281] INFO Started AdminServer on address 0.0.0.0, port 8080 and command URL /commands (org.apache.zookeeper.server.admin.JettyAdminServer)
[2021-03-24 19:03:09,288] INFO Election port bind maximum retries is 3 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
[2021-03-24 19:03:09,290] INFO 1 is accepting connections now, my election bind port: devkafka04/172.16.87.141:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
[2021-03-24 19:03:09,301] INFO LOOKING (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,303] INFO New election. My id = 1, proposed zxid=0x1600000030 (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2021-03-24 19:03:09,308] INFO Notification: 2 (message format version), 1 (n.leader), 0x1600000030 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x16 (n.peerEPoch), LOOKING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2021-03-24 19:03:09,310] INFO Have smaller server identifier, so dropping the connection: (myId:1 --> sid:3) (org.apache.zookeeper.server.quorum.QuorumCnxManager)
[2021-03-24 19:03:09,312] INFO Received connection request from /172.16.87.143:53340 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
[2021-03-24 19:03:09,315] INFO Have smaller server identifier, so dropping the connection: (myId:1 --> sid:2) (org.apache.zookeeper.server.quorum.QuorumCnxManager)
[2021-03-24 19:03:09,316] INFO Notification: 2 (message format version), 2 (n.leader), 0x150000002b (n.zxid), 0xa (n.round), FOLLOWING (n.state), 3 (n.sid), 0x16 (n.peerEPoch), LOOKING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2021-03-24 19:03:09,317] INFO Received connection request from /172.16.87.142:51704 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
[2021-03-24 19:03:09,319] INFO Notification: 2 (message format version), 2 (n.leader), 0x150000002b (n.zxid), 0xa (n.round), LEADING (n.state), 2 (n.sid), 0x16 (n.peerEPoch), LOOKING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2021-03-24 19:03:09,320] INFO Notification: 2 (message format version), 2 (n.leader), 0x150000002b (n.zxid), 0xa (n.round), FOLLOWING (n.state), 3 (n.sid), 0x16 (n.peerEPoch), LOOKING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2021-03-24 19:03:09,320] INFO FOLLOWING (org.apache.zookeeper.server.quorum.QuorumPeer)
[2021-03-24 19:03:09,323] INFO Notification: 2 (message format version), 2 (n.leader), 0x150000002b (n.zxid), 0xa (n.round), LEADING (n.state), 2 (n.sid), 0x16 (n.peerEPoch), FOLLOWING (my state)0 (n.config version) (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2021-03-24 19:03:09,330] INFO TCP NoDelay set to: true (org.apache.zookeeper.server.quorum.Learner)
[2021-03-24 19:03:09,336] INFO Server environment:zookeeper.version=3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:host.name=devkafka04 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:java.version=11.0.9.1 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:java.vendor=Azul Systems, Inc. (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:java.home=/usr/lib/jvm/zulu11-ca (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:java.class.path=/usr/bin/../share/java/kafka/activation-1.1.1.jar:/usr/bin/../share/java/kafka/aopalliance-repackaged-2.6.1.jar:/usr/bin/../share/java/kafka/argparse4j-0.7.0.jar:/usr/bin/../share/java/kafka/audience-annotations-0.5.0.jar:/usr/bin/../share/java/kafka/commons-cli-1.4.jar:/usr/bin/../share/java/kafka/commons-lang3-3.8.1.jar:/usr/bin/../share/java/kafka/confluent-log4j-1.2.17-cp2.jar:/usr/bin/../share/java/kafka/connect-api-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/connect-basic-auth-extension-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/connect-file-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/connect-json-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/connect-mirror-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/connect-mirror-client-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/connect-runtime-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/connect-transforms-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/hk2-api-2.6.1.jar:/usr/bin/../share/java/kafka/hk2-locator-2.6.1.jar:/usr/bin/../share/java/kafka/hk2-utils-2.6.1.jar:/usr/bin/../share/java/kafka/jackson-annotations-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-core-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-databind-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-dataformat-csv-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-datatype-jdk8-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-base-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-json-provider-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-module-jaxb-annotations-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-module-paranamer-2.10.5.jar:/usr/bin/../share/java/kafka/jackson-module-scala_2.13-2.10.5.jar:/usr/bin/../share/java/kafka/jakarta.activation-api-1.2.1.jar:/usr/bin/../share/java/kafka/jakarta.annotation-api-1.3.5.jar:/usr/bin/../share/java/kafka/jakarta.inject-2.6.1.jar:/usr/bin/../share/java/kafka/jakarta.validation-api-2.0.2.jar:/usr/bin/../share/java/kafka/jakarta.ws.rs-api-2.1.6.jar:/usr/bin/../share/java/kafka/jakarta.xml.bind-api-2.3.2.jar:/usr/bin/../share/java/kafka/javassist-3.25.0-GA.jar:/usr/bin/../share/java/kafka/javassist-3.26.0-GA.jar:/usr/bin/../share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/bin/../share/java/kafka/javax.ws.rs-api-2.1.1.jar:/usr/bin/../share/java/kafka/jaxb-api-2.3.0.jar:/usr/bin/../share/java/kafka/jersey-client-2.30.jar:/usr/bin/../share/java/kafka/jersey-common-2.30.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-2.30.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-core-2.30.jar:/usr/bin/../share/java/kafka/jersey-hk2-2.30.jar:/usr/bin/../share/java/kafka/jersey-media-jaxb-2.30.jar:/usr/bin/../share/java/kafka/jersey-server-2.30.jar:/usr/bin/../share/java/kafka/jetty-client-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-continuation-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-http-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-io-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-security-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-server-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-servlet-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-servlets-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jetty-util-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jopt-simple-5.0.4.jar:/usr/bin/../share/java/kafka/kafka-clients-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/kafka-log4j-appender-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/kafka-streams-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/kafka-streams-examples-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/kafka-streams-scala_2.13-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/kafka-streams-test-utils-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/kafka-tools-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/kafka.jar:/usr/bin/../share/java/kafka/kafka_2.13-6.0.1-ccs-javadoc.jar:/usr/bin/../share/java/kafka/kafka_2.13-6.0.1-ccs-scaladoc.jar:/usr/bin/../share/java/kafka/kafka_2.13-6.0.1-ccs-sources.jar:/usr/bin/../share/java/kafka/kafka_2.13-6.0.1-ccs-test-sources.jar:/usr/bin/../share/java/kafka/kafka_2.13-6.0.1-ccs-test.jar:/usr/bin/../share/java/kafka/kafka_2.13-6.0.1-ccs.jar:/usr/bin/../share/java/kafka/lz4-java-1.7.1.jar:/usr/bin/../share/java/kafka/maven-artifact-3.6.3.jar:/usr/bin/../share/java/kafka/metrics-core-2.2.0.jar:/usr/bin/../share/java/kafka/netty-buffer-4.1.50.Final.jar:/usr/bin/../share/java/kafka/netty-codec-4.1.50.Final.jar:/usr/bin/../share/java/kafka/netty-common-4.1.50.Final.jar:/usr/bin/../share/java/kafka/netty-handler-4.1.50.Final.jar:/usr/bin/../share/java/kafka/netty-resolver-4.1.50.Final.jar:/usr/bin/../share/java/kafka/netty-transport-4.1.50.Final.jar:/usr/bin/../share/java/kafka/netty-transport-native-epoll-4.1.50.Final.jar:/usr/bin/../share/java/kafka/netty-transport-native-unix-common-4.1.50.Final.jar:/usr/bin/../share/java/kafka/osgi-resource-locator-1.0.3.jar:/usr/bin/../share/java/kafka/paranamer-2.8.jar:/usr/bin/../share/java/kafka/plexus-utils-3.2.1.jar:/usr/bin/../share/java/kafka/reflections-0.9.12.jar:/usr/bin/../share/java/kafka/rocksdbjni-5.18.4.jar:/usr/bin/../share/java/kafka/scala-collection-compat_2.13-2.1.6.jar:/usr/bin/../share/java/kafka/scala-java8-compat_2.13-0.9.1.jar:/usr/bin/../share/java/kafka/scala-library-2.13.2.jar:/usr/bin/../share/java/kafka/slf4j-api-1.7.30.jar:/usr/bin/../share/java/kafka/scala-logging_2.13-3.9.2.jar:/usr/bin/../share/java/kafka/scala-reflect-2.13.2.jar:/usr/bin/../share/java/kafka/slf4j-log4j12-1.7.30.jar:/usr/bin/../share/java/kafka/snappy-java-1.1.7.3.jar:/usr/bin/../share/java/kafka/zookeeper-3.5.8.jar:/usr/bin/../share/java/kafka/zookeeper-jute-3.5.8.jar:/usr/bin/../share/java/kafka/zstd-jni-1.4.4-7.jar:/usr/bin/../share/java/confluent-telemetry/* (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:java.io.tmpdir=/tmp (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,336] INFO Server environment:java.compiler=<NA> (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:os.name=Linux (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:os.version=3.10.0-1160.21.1.el7.x86_64 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:user.name=appuser (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:user.home=/home/appuser (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:user.dir=/home/appuser (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:os.memory.free=498MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:os.memory.max=512MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,337] INFO Server environment:os.memory.total=512MB (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,338] INFO minSessionTimeout set to 6000 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,339] INFO maxSessionTimeout set to 60000 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,339] INFO Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 60000 datadir /var/lib/zookeeper/log/version-2 snapdir /var/lib/zookeeper/data/version-2 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-03-24 19:03:09,339] INFO FOLLOWING - LEADER ELECTION TOOK - 18 MS (org.apache.zookeeper.server.quorum.Learner)
[2021-03-24 19:03:09,345] INFO Getting a diff from the leader 0x1600000030 (org.apache.zookeeper.server.quorum.Learner)
[2021-03-24 19:03:09,350] INFO Learner received NEWLEADER message (org.apache.zookeeper.server.quorum.Learner)
[2021-03-24 19:03:09,363] INFO Learner received UPTODATE message (org.apache.zookeeper.server.quorum.Learner)
[2021-03-24 19:03:09,367] INFO Configuring CommitProcessor with 4 worker threads. (org.apache.zookeeper.server.quorum.CommitProcessor)
$ sudo docker logs kafka
===> User
uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
===> Configuring ...
SSL is enabled.
SASL is enabled.
===> Running preflight checks ...
===> Check if /var/lib/kafka/data is writable ...
===> Skipping Zookeeper health check for SSL connections...
===> Launching ...
===> Launching kafka ...
[2021-03-23 21:43:43,453] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2021-03-23 21:43:43,838] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2021-03-23 21:43:43,900] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2021-03-23 21:43:43,904] INFO starting (kafka.server.KafkaServer)
[2021-03-23 21:43:43,905] INFO Connecting to zookeeper on devkafka04:2182,devkafka05:2182,devkafka06:2182 (kafka.server.KafkaServer)
[2021-03-23 21:43:43,927] INFO [ZooKeeperClient Kafka server] Initializing a new session to devkafka04:2182,devkafka05:2182,devkafka06:2182. (kafka.zookeeper.ZooKeeperClient)
[2021-03-23 21:43:43,934] INFO Client environment:zookeeper.version=3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT (org.apache.zookeeper.ZooKeeper)
[2021-03-23 21:43:43,934] INFO Client environment:host.name=devkafka04 (org.apache.zookeeper.ZooKeeper)
[2021-03-23 21:43:43,934] INFO Client environment:java.version=11.0.9.1 (org.apache.zookeeper.ZooKeeper)
[2021-03-23 21:43:43,934] INFO Client environment:java.vendor=Azul Systems, Inc. (org.apache.zookeeper.ZooKeeper)
------ Repeating lines removed ---------
'Client' (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:43:59,947] INFO Socket error occurred: devkafka05/172.16.87.142:2182: Connection refused (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:01,048] INFO Client successfully logged in. (org.apache.zookeeper.Login)
[2021-03-23 21:44:01,048] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2021-03-23 21:44:01,048] INFO Opening socket connection to server devkafka04/172.16.87.141:2182. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:01,049] INFO Socket error occurred: devkafka04/172.16.87.141:2182: Connection refused (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:01,150] INFO Client successfully logged in. (org.apache.zookeeper.Login)
[2021-03-23 21:44:01,150] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2021-03-23 21:44:01,150] INFO Opening socket connection to server devkafka06/172.16.87.143:2182. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:01,153] INFO Socket error occurred: devkafka06/172.16.87.143:2182: Connection refused (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:01,254] INFO Client successfully logged in. (org.apache.zookeeper.Login)
[2021-03-23 21:44:01,254] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2021-03-23 21:44:01,254] INFO Opening socket connection to server devkafka05/172.16.87.142:2182. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:01,255] INFO Socket error occurred: devkafka05/172.16.87.142:2182: Connection refused (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:01,952] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2021-03-23 21:44:02,356] INFO Client successfully logged in. (org.apache.zookeeper.Login)
[2021-03-23 21:44:02,357] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2021-03-23 21:44:02,357] INFO Opening socket connection to server devkafka04/172.16.87.141:2182. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:02,462] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2021-03-23 21:44:02,463] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2021-03-23 21:44:02,465] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2021-03-23 21:44:02,469] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:262)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:119)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1865)
at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:419)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:444)
at kafka.server.KafkaServer.startup(KafkaServer.scala:222)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
[2021-03-23 21:44:02,471] INFO shutting down (kafka.server.KafkaServer)
[2021-03-23 21:44:02,478] INFO shut down completed (kafka.server.KafkaServer)
[2021-03-23 21:44:02,478] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2021-03-23 21:44:02,479] INFO shutting down (kafka.server.KafkaServer)
$ sudo cat kafka-docker-compose.yml
version: '3'
services:
kafka:
image: confluentinc/cp-kafka:6.0.1
container_name: kafka
network_mode: host
restart: always
ports:
- "9092:9092"
- "9093:9093"
- "9094:9094"
- "49998:49998"
- "49999:49999"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'devkafka04:2182,devkafka05:2182,devkafka06:2182'
KAFKA_ZOOKEEPER_SSL_CLIENT_ENABLE: 'true'
KAFKA_ZOOKEEPER_CLIENTCNXNSOCKET: org.apache.zookeeper.ClientCnxnSocketNetty
KAFKA_ZOOKEEPER_SSL_TRUSTSTORE_FILENAME: kafka.server.truststore.jks
KAFKA_ZOOKEEPER_SSL_TRUSTSTORE_CREDENTIALS: creds
KAFKA_ZOOKEEPER_SET_ACL: 'true'
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://devkafka04:9092,SSL://devkafka04:9093,SASL_SSL://devkafka04:9094
KAFKA_LISTENERS: PLAINTEXT://devkafka04:9092,SSL://devkafka04:9093,SASL_SSL://devkafka04:9094
KAFKA_SASL_ENABLED_MECHANISMS: DIGEST-MD5
KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SSL
KAFKA_SSL_CLIENT_AUTH: requested
KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.server.truststore.jks
KAFKA_SSL_TRUSTSTORE_CREDENTIALS: creds
KAFKA_SSL_KEYSTORE_FILENAME: devkafka04.server.keystore.jks
KAFKA_SSL_KEYSTORE_CREDENTIALS: creds
KAFKA_SSL_KEY_CREDENTIALS: creds
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/jmx/kafka_server_jaas.conf -Djava.rmi.server.hostname=devkafka04 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.rmi.port=49998 -Dcom.sun.management.jmxremote.port=49998 -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -javaagent:/etc/kafka/jmx/jmx_prometheus_javaagent-0.14.0.jar=49999:/etc/kafka/jmx/kafka-2_0_0.yml
CONFLUENT_SUPPORT_METRICS_ENABLE: "false"
volumes:
- /media/kafka/data:/var/lib/kafka/data
- /media/kafka/secrets:/etc/kafka/secrets
- /usr/local/src/kafka/jmx:/etc/kafka/jmx
$ sudo cat jmx/kafka_server_jaas.conf
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="kafkabroker"
password="kafkabroker-secret"
user_kafkabroker="kafkabroker-secret"
user_kafka-broker-metric-reporter="kafkabroker-metric-reporter-secret"
user_client="client-secret";
};
Client {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="kafka"
password="kafka-secret";
};
$ sudo cat zookeeper-docker-compose.yml
version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:6.0.1
container_name: zookeeper
network_mode: host
restart: always
ports:
- "2181:2181"
- "2182:2182"
- "2888:2888"
- "3888:3888"
- "39998:39998"
- "39999:39999"
environment:
ZOOKEEPER_SERVER_ID: 1
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_SERVERS: devkafka04:2888:3888;devkafka05:2888:3888;devkafka06:2888:3888
ZOOKEEPER_AUTHPROVIDER_SASL: org.apache.zookeeper.server.auth.SASLAuthenticationProvider
ZOOKEEPER_AUTHPROVIDER_x509: org.apache.zookeeper.server.auth.X509AuthenticationProvider
ZOOKEEPER_SECURECLIENTPORT: 2182
ZOOKEEPER_SERVERCNXNFACTORY: org.apache.zookeeper.server.NettyServerCnxnFactory
ZOOKEEPER_SSL_TRUSTSTORE_FILENAME: kafka.server.truststore.jks
ZOOKEEPER_SSL_TRUSTSTORE_CREDENTIALS: creds
ZOOKEEPER_SSL_KEYSTORE_FILENAME: devkafka05.server.keystore.jks
ZOOKEEPER_SSL_KEYSTORE_CREDENTIALS: creds
ZOOKEEPER_SSL_KEY_CREDENTIALS: creds
ZOOKEEPER_SSL_CLIENTAUTH: none
KAFKA_OPTS: -Djava.security.auth.login.config=/etc/zookeeper/jmx/zookeeper_jaas.conf -Dzookeeper.4lw.commands.whitelist=* -Djava.rmi.server.hostname=devkafka04 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.rmi.port=39998 -Dcom.sun.management.jmxremote.port=39998 -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -javaagent:/etc/zookeeper/jmx/jmx_prometheus_javaagent-0.14.0.jar=39999:/etc/zookeeper/jmx/jmx-zookeeper-prometheus.yaml
volumes:
- /media/zookeeper/data:/var/lib/zookeeper/data
- /media/zookeeper/log:/var/lib/zookeeper/log
- /media/zookeeper/secrets:/etc/zookeeper/secrets
- /usr/local/src/zookeeper/jmx:/etc/zookeeper/jmx
$ sudo cat jmx/zookeeper_jaas.conf
Server {
org.apache.zookeeper.server.auth.DigestLoginModule required
user_kafka="kafka-secret";
};
Try using KAFKA_ZOOKEEPER_CLIENT_CNXN_SOCKET instead of KAFKA_ZOOKEEPER_CLIENTCNXNSOCKET.
I am trying to install kafka in ubuntu. I have downloaded the kafka tar.gz file,unzipped it. started the zookeeper server .While trying to start the kafka server, getting the timeout exception.
Can some one pls let me know the resolution.
Following are the server logs:
kafka-server-start.sh ../config/server.properties
[2020-09-17 15:53:02,990] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-09-17 15:53:03,310] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2020-09-17 15:53:03,347] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2020-09-17 15:53:03,351] INFO starting (kafka.server.KafkaServer)
[2020-09-17 15:53:03,352] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2020-09-17 15:53:03,368] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2020-09-17 15:53:03,373] INFO Client environment:zookeeper.version=3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,373] INFO Client environment:host.name=LAPTOP-D6RGQ60T.localdomain (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,373] INFO Client environment:java.version=1.8.0_261 (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,373] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,373] INFO Client environment:java.home=/opt/jdk/jre1.8.0_261 (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,373] INFO Client environment:java.class.path=/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/activation-1.1.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/aopalliance-repackaged-2.5.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/argparse4j-0.7.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/audience-annotations-0.5.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/commons-cli-1.4.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/commons-lang3-3.8.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-api-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-basic-auth-extension-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-file-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-json-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-mirror-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-mirror-client-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-runtime-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/connect-transforms-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/hk2-api-2.5.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/hk2-locator-2.5.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/hk2-utils-2.5.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-annotations-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-core-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-databind-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-dataformat-csv-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-datatype-jdk8-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-jaxrs-base-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-jaxrs-json-provider-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-module-jaxb-annotations-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-module-paranamer-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jackson-module-scala_2.12-2.10.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jakarta.activation-api-1.2.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jakarta.annotation-api-1.3.4.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jakarta.inject-2.5.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jakarta.ws.rs-api-2.1.5.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/javassist-3.22.0-CR2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/javassist-3.26.0-GA.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/javax.servlet-api-3.1.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/javax.ws.rs-api-2.1.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jaxb-api-2.3.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jersey-client-2.28.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jersey-common-2.28.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jersey-container-servlet-2.28.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jersey-container-servlet-core-2.28.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jersey-hk2-2.28.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jersey-media-jaxb-2.28.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jersey-server-2.28.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-client-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-continuation-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-http-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-io-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-security-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-server-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-servlet-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-servlets-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jetty-util-9.4.24.v20191120.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/jopt-simple-5.0.4.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka-clients-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka-log4j-appender-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka-streams-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka-streams-examples-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka-streams-scala_2.12-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka-streams-test-utils-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka-tools-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka_2.12-2.6.0-sources.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/kafka_2.12-2.6.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/log4j-1.2.17.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/lz4-java-1.7.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/maven-artifact-3.6.3.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/metrics-core-2.2.0.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-buffer-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-codec-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-common-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-handler-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-resolver-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-transport-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-transport-native-epoll-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/netty-transport-native-unix-common-4.1.50.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/osgi-resource-locator-1.0.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/paranamer-2.8.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/plexus-utils-3.2.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/reflections-0.9.12.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/rocksdbjni-5.18.4.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/scala-collection-compat_2.12-2.1.6.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/scala-java8-compat_2.12-0.9.1.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/scala-library-2.12.11.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/scala-logging_2.12-3.9.2.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/scala-reflect-2.12.11.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/slf4j-api-1.7.30.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/slf4j-log4j12-1.7.30.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/snappy-java-1.1.7.3.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/validation-api-2.0.1.Final.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/zookeeper-3.5.8.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/zookeeper-jute-3.5.8.jar:/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin/../libs/zstd-jni-1.4.4-7.jar (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:os.version=4.4.0-18362-Microsoft (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:user.name=sreekanth (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:user.home=/home/sreekanth (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:user.dir=/home/sreekanth/Downloads/kafka_2.12-2.6.0/bin (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:os.memory.free=976MB (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,374] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,376] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$#47af7f3d (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:53:03,380] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2020-09-17 15:53:03,384] INFO zookeeper.request.timeout value is 0. feature enabled= (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:03,386] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2020-09-17 15:53:03,389] INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:03,392] INFO Socket connection established, initiating session, client: /127.0.0.1:50975, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:21,394] WARN Client session timed out, have not heard from server in 18001ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:21,394] INFO Client session timed out, have not heard from server in 18001ms for sessionid 0x0, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:23,114] INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:23,115] INFO Socket connection established, initiating session, client: /127.0.0.1:50976, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:41,117] WARN Client session timed out, have not heard from server in 18001ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:41,117] INFO Client session timed out, have not heard from server in 18001ms for sessionid 0x0, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:42,712] INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:53:42,713] INFO Socket connection established, initiating session, client: /127.0.0.1:50977, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:54:00,714] WARN Client session timed out, have not heard from server in 18001ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:54:00,715] INFO Client session timed out, have not heard from server in 18001ms for sessionid 0x0, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:54:02,432] INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:54:02,433] INFO Socket connection established, initiating session, client: /127.0.0.1:50980, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:54:03,388] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2020-09-17 15:54:20,433] WARN Client session timed out, have not heard from server in 18000ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:54:20,555] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2020-09-17 15:54:20,556] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2020-09-17 15:54:20,557] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2020-09-17 15:54:20,559] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:262)
at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:258)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:119)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1865)
at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:419)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:444)
at kafka.server.KafkaServer.startup(KafkaServer.scala:222)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
[2020-09-17 15:54:20,561] INFO shutting down (kafka.server.KafkaServer)
[2020-09-17 15:54:20,565] INFO shut down completed (kafka.server.KafkaServer)
[2020-09-17 15:54:20,566] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2020-09-17 15:54:20,567] INFO shutting down (kafka.server.KafkaServer)
Many Zookeeper instances were running earlier. I killed all the zookeeper and Brokers , restarted them again freshly . It is working fine now.
I am very new to using Microservices and having trouble running Kafka after I have started zookeeper.
Zookeeper starts fine but when I try to start my Kafka server it throws an error.
I have searched on google to try and solve my problem but its quite overwhelming, as I am not sure what all these different config files mean/do.
I have tried by enabling listeners=PLAINTEXT://:9092 in server settings but it doesn't work.
I have also tried to un and reinstalled Kafka and ZooKeeper but I still get the same error.
Antonias-MacBook-Pro:kafka antonia$ kafka-server-start /usr/local/etc/kafka/server.properties
[2020-02-25 10:26:55,944] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-02-25 10:26:56,514] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2020-02-25 10:26:56,514] INFO starting (kafka.server.KafkaServer)
[2020-02-25 10:26:56,515] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2020-02-25 10:26:56,541] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:56,571] INFO Client environment:zookeeper.version=3.5.6-c11b7e26bc554b8523dc929761dd28808913f091, built on 10/08/2019 20:18 GMT (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:host.name=10.111.180.36 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.version=1.8.0_242 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.vendor=AdoptOpenJDK (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.home=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.class.path=/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/activation-1.1.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/aopalliance-repackaged-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/argparse4j-0.7.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/audience-annotations-0.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/commons-cli-1.4.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/commons-lang3-3.8.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-api-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-basic-auth-extension-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-file-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-json-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-mirror-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-mirror-client-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-runtime-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-transforms-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/guava-20.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/hk2-api-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/hk2-locator-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/hk2-utils-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-annotations-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-core-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-databind-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-dataformat-csv-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-datatype-jdk8-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-jaxrs-base-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-jaxrs-json-provider-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-module-jaxb-annotations-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-module-paranamer-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-module-scala_2.12-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.activation-api-1.2.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.annotation-api-1.3.4.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.inject-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.ws.rs-api-2.1.5.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/javassist-3.22.0-CR2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/javax.servlet-api-3.1.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/javax.ws.rs-api-2.1.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jaxb-api-2.3.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-client-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-common-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-container-servlet-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-container-servlet-core-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-hk2-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-media-jaxb-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-server-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-client-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-continuation-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-http-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-io-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-security-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-server-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-servlet-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-servlets-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-util-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jopt-simple-5.0.4.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-clients-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-log4j-appender-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-examples-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-scala_2.12-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-test-utils-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-tools-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka_2.12-2.4.0-sources.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka_2.12-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/log4j-1.2.17.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/lz4-java-1.6.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/maven-artifact-3.6.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/metrics-core-2.2.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-buffer-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-codec-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-common-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-handler-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-resolver-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-transport-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-transport-native-epoll-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-transport-native-unix-common-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/osgi-resource-locator-1.0.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/paranamer-2.8.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/plexus-utils-3.2.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/reflections-0.9.11.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/rocksdbjni-5.18.3.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-collection-compat_2.12-2.1.2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-java8-compat_2.12-0.9.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-library-2.12.10.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-logging_2.12-3.9.2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-reflect-2.12.10.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/slf4j-api-1.7.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/slf4j-log4j12-1.7.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/snappy-java-1.1.7.3.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/validation-api-2.0.1.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/zookeeper-3.5.6.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/zookeeper-jute-3.5.6.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/zstd-jni-1.4.3-1.jar (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:java.library.path=/Users/antonia/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:. (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:java.io.tmpdir=/var/folders/13/gvhn5dqj4179ynmpm6pkp9kh0000gn/T/ (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:os.name=Mac OS X (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:os.arch=x86_64 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.version=10.14.5 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:user.name=antonia (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:user.home=/Users/antonia (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:user.dir=/usr/local/etc/kafka (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.memory.free=977MB (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,575] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$#28eaa59a (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,582] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2020-02-25 10:26:56,592] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2020-02-25 10:26:56,599] INFO zookeeper.request.timeout value is 0. feature enabled= (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,601] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:56,605] INFO Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,625] INFO Socket connection established, initiating session, client: /0:0:0:0:0:0:0:1:63152, server: localhost/0:0:0:0:0:0:0:1:2181 (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,641] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x10004969600000a, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,645] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:56,979] INFO Cluster ID = S4SZ31nVRTCQ4uwRJ9_7mg (kafka.server.KafkaServer)
[2020-02-25 10:26:56,992] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID S4SZ31nVRTCQ4uwRJ9_7mg doesn't match stored clusterId Some(Y_mQi4q4TSuhlWdx4DHiaQ) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
at kafka.server.KafkaServer.startup(KafkaServer.scala:220)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:84)
at kafka.Kafka.main(Kafka.scala)
[2020-02-25 10:26:56,995] INFO shutting down (kafka.server.KafkaServer)
[2020-02-25 10:26:56,997] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:57,109] INFO Session: 0x10004969600000a closed (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:57,109] INFO EventThread shut down for session: 0x10004969600000a (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:57,111] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:57,118] INFO shut down completed (kafka.server.KafkaServer)
[2020-02-25 10:26:57,119] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2020-02-25 10:26:57,120] INFO shutting down (kafka.server.KafkaServer)
The cause of the problem is shown in this message:
kafka.common.InconsistentClusterIdException:
The Cluster ID S4SZ31nVRTCQ4uwRJ9_7mg
doesn't match stored clusterId Some(Y_mQi4q4TSuhlWdx4DHiaQ)
in meta.properties.
The broker is trying to join the wrong cluster.
Configured zookeeper.connect may be wrong.
The above problem occurs when a new instance of Kafka is being started up on data storage created by another kafka server. Kafka stores its messages in 'log' files.
How to fix the problem?
The problem can be fixed in these steps:
Shutdown both Kafka and Zookeeper
If required, take backup of the existing logs of Kafka and Zookeeper
Delete the log directories of both Kafka and Zookeeper
Restart Zookeeper and Kafka
Zookeeper is accepting a socket connection from 50167(in my case) and then closed it.
[2017-09-04 14:44:23,926] INFO Server environment:user.dir=C:\kafka_2.11-0.11.0.0 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-09-04 14:44:24,013] INFO tickTime set to 3000 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-09-04 14:44:24,029] INFO minSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-09-04 14:44:24,045] INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-09-04 14:44:24,245] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2017-09-04 14:45:16,525] INFO Accepted socket connection from /0:0:0:0:0:0:0:1:50167 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2017-09-04 14:45:16,557] INFO Client attempting to establish new session at /0:0:0:0:0:0:0:1:50167 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-09-04 14:45:16,572] INFO Creating new log file: log.b1 (org.apache.zookeeper.server.persistence.FileTxnLog)
[2017-09-04 14:45:16,613] INFO Established session 0x15e4c2b5f7f0000 with negotiated timeout 6000 for client /0:0:0:0:0:0:0:1:50167 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-09-04 14:45:17,939] INFO Processed session termination for sessionid: 0x15e4c2b5f7f0000 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-09-04 14:45:17,970] INFO Closed socket connection for client /0:0:0:0:0:0:0:1:50167 which had sessionid 0x15e4c2b5f7f0000 (org.apache.zookeeper.server.NIOServerCnxn)
Due to this, Kafka server is also failing to run. Any solution will be helpful.
After, I deleted the old logs at location:
i.e. folders kafka_2.11-0.11.0.0kafka-logs & kafka_2.11-0.11.0.0zookeeper-data, the server started fine.
I have just started learning Kafka/Zookeper. Issue I am facing is, when I try to run the console consumer to verify the message for a topic, I get the below error-
vagrant#debian-70rc1-x64-vbox4210:~$ $KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper 192.168.33.10:2181 --topic topic-3
[2015-02-22 22:56:36,666] INFO Verifying properties (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,698] INFO Property group.id is overridden to console-consumer-92151 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,699] INFO Property fetch.message.max.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,700] INFO Property consumer.timeout.ms is overridden to -1 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,700] INFO Property socket.timeout.ms is overridden to 30000 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,700] INFO Property auto.offset.reset is overridden to largest (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,701] INFO Property zookeeper.connect is overridden to 192.168.33.10:2181 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,701] INFO Property socket.receive.buffer.bytes is overridden to 2097152 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,701] INFO Property fetch.wait.max.ms is overridden to 100 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,702] INFO Property auto.commit.enable is overridden to true (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,702] INFO Property refresh.leader.backoff.ms is overridden to 200 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,702] INFO Property auto.commit.interval.ms is overridden to 60000 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,703] INFO Property fetch.min.bytes is overridden to 1 (kafka.utils.VerifiableProperties)
[2015-02-22 22:56:36,734] INFO [console-consumer-92151_debian-70rc1-x64-vbox4210-1424645796729-11446a89], Connecting to zookeeper instance at 192.168.33.10:2181 (kafka.consumer.ZookeeperConsumerConnector)
[2015-02-22 22:56:36,754] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-02-22 22:56:36,755] INFO Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,756] INFO Client environment:host.name=debian-70rc1-x64-vbox4210.vagrantup.com (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,756] INFO Client environment:java.version=1.7.0_75 (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,757] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,757] INFO Client environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,757] INFO Client environment:java.class.path=:/usr/local/kafka/kafka-0.8.0-beta1-src//bin/../core/target/scala-2.8.0/kafka_2.8.0-0.8.0-beta1.jar:/usr/local/kafka/kafka-0.8.0-beta1-src//bin/../core/target/scala-2.8.0/kafka-assembly-0.8.0-beta1-deps.jar:/usr/local/kafka/kafka-0.8.0-beta1-src//bin/../perf/target/scala-2.8.0/kafka-perf_2.8.0-0.8.0-beta1.jar:/usr/local/kafka/kafka-0.8.0-beta1-src//bin/../libs/*.jar:/usr/local/kafka/kafka-0.8.0-beta1-src//bin/../kafka*.jar (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,758] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,758] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,758] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,759] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,759] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,759] INFO Client environment:os.version=3.2.0-4-amd64 (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,759] INFO Client environment:user.name=vagrant (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,760] INFO Client environment:user.home=/home/vagrant (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,760] INFO Client environment:user.dir=/home/vagrant (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,761] INFO Initiating client connection, connectString=192.168.33.10:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient#234a98fa (org.apache.zookeeper.ZooKeeper)
[2015-02-22 22:56:36,773] INFO Opening socket connection to server /192.168.33.10:2181 (org.apache.zookeeper.ClientCnxn)
[2015-02-22 22:56:42,778] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-02-22 22:56:46,791] INFO Socket connection established to 192.168.33.10/192.168.33.10:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-02-22 22:56:46,801] INFO Session establishment complete on server 192.168.33.10/192.168.33.10:2181, sessionid = 0x14bb3752996000d, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2015-02-22 22:56:46,803] INFO Session: 0x14bb3752996000d closed (org.apache.zookeeper.ZooKeeper)
Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:151)
at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:112)
at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:123)
at kafka.consumer.Consumer$.create(ConsumerConnector.scala:89)
at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:178)
at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
[2015-02-22 22:56:46,804] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn)
It says-
Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
I verified the topic and it is present. What is going wrong. Please help.
I had the same problem and here is how I fixed it:
1) Stop all Kafka and Zookeeper processes
ps -aux | grep zoo
ps -aux | grep kafka
(then proceed to kill all process ids from above jobs)
2) Run zookeeper
<kafka_dir>/bin/zookeeper-server-start.sh ../config/zookeeper.properties
3) Run kafka server
<kafka_dir>/bin/kafka-server-start.sh ../config/server.properties
Try -> telnet 192.168.33.10 2181
If the telnet fails, then the issue is due to below reason.
Zookeeper Server NRPE iptables chain does not appear to a have a default accept rule.
Unless you just want your own iptables chain for NRPE, you could use:
iptables -I INPUT -s 0/0 -p tcp --dport 2181 -j ACCEPT
This solves the issue.
You can try the follwoing to see if it helps ..
1) Make sure the zookeeper is up and running, try to connect to it using the zkCli client .
2) append --from-beginning at the end of your command to see if you get any diifference in the output , so the command should be changed to :
$KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper xxx.xx.xx.xxx:2181 --topic topic-3 --from-beginning
In my case it was already connected to the host from which I was trying, so it was not able to make any new connection to the host. I had to change the config: maxClientCnxns so that it was able to make more connection to that host. More details here.