While I am trying to start zookeeper I am getting below error
[2017-05-24 11:52:31,633] ERROR Last transaction was partial. (org.apache.zookeeper.server.persistence.Util)
[2017-05-24 11:52:31,634] ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain)
java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
at org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:158)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283)
at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:410)
at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118)
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:119)
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87)
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Could you please help me with this fix.
Thank you.
This sometimes happens when the snapshots in zookeeper dataDir is corrupt.
you can fix this by cleanup the contents of dataDir and restart zookeeper
Related
I just installed Kafka using brew install kafka and it was successful. Now when I try to start zookeeper I get the following error:
~/ zookeeper-server-start /opt/homebrew/etc/kafka/server.properties
[2022-08-17 12:03:46,961] INFO Reading configuration from: /opt/homebrew/etc/kafka/server.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2022-08-17 12:03:46,964] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /opt/homebrew/etc/kafka/server.properties
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:198)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:124)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:90)
Caused by: java.lang.IllegalArgumentException: dataDir is not set
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:444)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:194)
... 2 more
Invalid config, exiting abnormally
[2022-08-17 12:03:46,965] INFO ZooKeeper audit is disabled. (org.apache.zookeeper.audit.ZKAuditProvider)
[2022-08-17 12:03:46,966] ERROR Exiting JVM with code 2 (org.apache.zookeeper.util.ServiceUtils)
~/
The server.properties file exists:
-rw-r--r-- 1 atael admin 6912 Aug 17 11:51 /opt/homebrew/etc/kafka/server.properties
My environment is a M1 Mac if that matters.
Anyone seems this before and can help?
Thanks
Andy
Found the problem. The right command should be zookeeper-server-start /opt/homebrew/etc/kafka/zookeeper.properties
ERROR Error while creating ephemeral at /brokers/ids/0, node already exists and owner '72067757872119809' does not match current session '72067836689711106' (kafka.zk.KafkaZkClient$CheckedEphemeral) 2021-05-05 02:19:44.796 [INF] [Kafka] [2021-05-05 02:19:44,786] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
The error is saying a broker already is running with id=0, or that Zookeeper is corrupt because a broker did not previously cleanly shut down...
In the later case, you can attempt to use zookeeper-shell to rmr /brokers/ids/0, however, this might have more unintended consequences than preforming a restart of Zookeeper as well as the brokers
The only solution which works here is restart the broker and then restart the server.
Restart the zookeeper and the broker is fixed the issue for me.
If you using the docker-compose, you can restart simply by using
docker-compose restart
Thanks in advance. Please help me to resolve this below mentioned kafka error.
00000000000000.txnindex and rebuilding index... (kafka.log.Log)
[2018-09-25 12:48:05,462] ERROR There was an error in one of the threads during logs loading: java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code (kafka.log.LogManager)
[2018-09-25 12:48:05,469] FATAL [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
at org.apache.kafka.common.record.FileLogInputStream$FileChannelRecordBatch.loadBatchWithSize(FileLogInputStream.java:209)
at org.apache.kafka.common.record.FileLogInputStream$FileChannelRecordBatch.loadFullBatch(FileLogInputStream.java:192)
at org.apache.kafka.common.record.FileLogInputStream$FileChannelRecordBatch.ensureValid(FileLogInputStream.java:164)
at kafka.log.LogSegment$$anonfun$recover$1.apply(LogSegment.scala:263)
at kafka.log.LogSegment$$anonfun$recover$1.apply(LogSegment.scala:262)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
My guess is that your Kafka log directory (log.dirs in server.properties) is out of memory. When Kafka attempts to rebuild the index, there is insufficient memory and therefore Kafka broker cannot be started.
Assuming log.dirs=/var/lib/kafka
df -hT /var/lib/kafka
will display storage usage for your log directory.
Kafka is throwing an exception while starting,
Timed out waiting for connection while in state: CONNECTING
Followed by null pointer exception.
I have increased the config/server.properties connection time out but still has the issue.
Here am using the windows Os.
Thanks
ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply$mcV$sp(ZooKeeperClient.scala:230)
at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:226)
at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:226)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251)
at kafka.zookeeper.ZooKeeperClient.kafka$zookeeper$ZooKeeperClient$$waitUntilConnected(ZooKeeperClient.scala:226)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:95)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1580)
at kafka.server.KafkaServer.kafka$server$KafkaServer$$createZkClient$1(KafkaServer.scala:348)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:372)
at kafka.server.KafkaServer.startup(KafkaServer.scala:202)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:75)
at kafka.Kafka.main(Kafka.scala)
[2018-09-04 23:31:35,864] INFO shutting down (kafka.server.KafkaServer)
After restarting the system it got started working.it did not work even changing the kafka server.properties, but its working now as my machine got started.
Thanks every one for helping me to debug.
I'm trying to start zookeeper server process through ambari in HDP. However, I'm getting error stating that Connection failed: [Errno 111].
I have checked in server logs (/var/log/zookeeper/server.out). It is giving below error. Can someone please help me to understand or resolve issue.
ERROR [ZooKeeperServer#294] - Severe unrecoverable error, exiting
java.io.IOException: No space left on devices.
The error is clear. You have no free Disk-Space on your Node where Zookeeper is running.
Check it with df -h if its a Linux-Server.