I just installed Kafka using brew install kafka and it was successful. Now when I try to start zookeeper I get the following error:
~/ zookeeper-server-start /opt/homebrew/etc/kafka/server.properties
[2022-08-17 12:03:46,961] INFO Reading configuration from: /opt/homebrew/etc/kafka/server.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2022-08-17 12:03:46,964] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /opt/homebrew/etc/kafka/server.properties
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:198)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:124)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:90)
Caused by: java.lang.IllegalArgumentException: dataDir is not set
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:444)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:194)
... 2 more
Invalid config, exiting abnormally
[2022-08-17 12:03:46,965] INFO ZooKeeper audit is disabled. (org.apache.zookeeper.audit.ZKAuditProvider)
[2022-08-17 12:03:46,966] ERROR Exiting JVM with code 2 (org.apache.zookeeper.util.ServiceUtils)
~/
The server.properties file exists:
-rw-r--r-- 1 atael admin 6912 Aug 17 11:51 /opt/homebrew/etc/kafka/server.properties
My environment is a M1 Mac if that matters.
Anyone seems this before and can help?
Thanks
Andy
Found the problem. The right command should be zookeeper-server-start /opt/homebrew/etc/kafka/zookeeper.properties
Related
I am new to kafka and i started doing the Zookeeper cluster setup by refering https://codeforgeek.com/how-to-setup-zookeeper-cluster-for-kafka/
This has three zookeeper instance, I am able to start the fist zookeeper instance but in the second node zookeeper is not started.
I used this command to start zookeeper - bin/zookeeper-server-start.sh config/zookeeper.properties
I also created myid file with number 2 written in it.
Throwing the below error,
[root#kafka2 kafka_2.12-2.2.2]# bin/zookeeper-server-start.sh config/zookeeper.properties
[2020-09-06 07:25:27,761] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2020-09-06 07:25:27,773] INFO Resolved hostname: 0.0.0.0 to address: /0.0.0.0 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2020-09-06 07:25:27,773] INFO Resolved hostname: 172.16.10.45 to address: /172.16.10.45 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2020-09-06 07:25:27,775] INFO Resolved hostname: 172.16.10.44 to address: /172.16.10.44 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2020-09-06 07:25:27,776] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing config/zookeeper.properties
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:156)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:104)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
Caused by: java.lang.IllegalArgumentException: Missing election port for server: 1
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:369)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:152)
... 2 more
Invalid config, exiting abnormally
I found the answers finally. Actually i missed to mention the port(2888) for server.1 in config/zookeeper.properties.
Issue
server.1=<ip of first server>:3888
server.2=0.0.0.0:2888:3888
server.3=<ip of third server>:2888:3888
Working
server.1=<ip of first server>:2888:3888
server.2=0.0.0.0:2888:3888
server.3=<ip of third server>:2888:3888
I am using a zookeeper ensemble of 3 nodes running 3.4.13. Sometimes after reboot of machine zookeeper is not starting in one of the node and I am seeing the below errors in logs
2019-08-19 04:18:36,906 [myid:2] - ERROR [main:QuorumPeer#692] - Unable to load database on disk
java.io.IOException: The current epoch, 7, is older than the last zxid, 34359738370
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:674)
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:635)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:170)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:114)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
2019-08-19 04:18:36,908 [myid:2] - ERROR [main:QuorumPeerMain#92] - Unexpected exception, exiting abnormally
java.lang.RuntimeException: Unable to run quorum server
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:693)
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:635)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:170)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:114)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
Caused by: java.io.IOException: The current epoch, 7, is older than the last zxid, 34359738370
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:674)
... 4 more----
I have seen ZOOKEEPER-2354 and the symptoms look similar.
support#platform2:/var/lib/zookeeper/version-2$ sudo cat acceptedEpoch
8support#platform2:/var/lib/zookeeper/version-2$ sudo cat currentEpoch
7support#platform2:/var/lib/zookeeper/version-2$ sudo cat currentEpoch.tmp
8support#platform2
The above issue states the issue is fixed in 3.4.6 but I am observing the same in 3.4.13.
Can someone let me know how can I recover the zookeeper node from this?
This has been discussed in zookeeper mailing thread. Relevant quote from that thread
With the other two zookeeper servers running I stopped the zookeeper
in the broken node and the deleted all the contents inside
/var/lib/zookeeper/version-2 and started the zookeeper back on the
node. It is running fine now and got all the data from the other
servers.
I am trying to start a Zookeper instance and i keep getting an error that says the zookeeper.properties file does not exist..when in fact it does.
$ ./script.sh [2019-06-30 07:40:00,844] INFO Reading configuration
from: .\config\zookepeer.properties
(org.apache.zookeeper.server.quorum.QuorumPeerConfig) [2019-06-30
07:40:00,844] ERROR Invalid config, exiting abnormally
(org.apache.zookeeper.server.quorum.QuorumPeerMain)
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException:
Error processing ./config/zookepeer.properties
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:156)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:104)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
Caused by: java.lang.IllegalArgumentException:
.\config\zookepeer.properties file is missing
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:140)
... 2 more Invalid config, exiting abnormally
Folder structure
-root
-config
-zookeeper.properties
-bin
-zookeeper-server-start.sh
-script.sh
Script
./bin/zookeeper-server-start.sh ./config/zookepeer.properties
Zookeeper.properties
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
Please check the properties file name in script.
./bin/zookeeper-server-start.sh ./config/zookepeer.properties .
Change it as,
./bin/zookeeper-server-start.sh ./config/zookeeper.properties
and check.
Try this
./zookeeper-server-start.sh ../config/zookeeper.properties
Whereby you are located in the bin directory.
While I am trying to start zookeeper I am getting below error
[2017-05-24 11:52:31,633] ERROR Last transaction was partial. (org.apache.zookeeper.server.persistence.Util)
[2017-05-24 11:52:31,634] ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain)
java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
at org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:158)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283)
at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:410)
at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118)
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:119)
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87)
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Could you please help me with this fix.
Thank you.
This sometimes happens when the snapshots in zookeeper dataDir is corrupt.
you can fix this by cleanup the contents of dataDir and restart zookeeper
After upgrading CDH5.3.2 to CDH5.3.5, zookeeper start failed, I got
Command aborted because of exception: Command timed-out after 150 seconds
in cloudera manager, and I got this error code in log file
2015-07-20 19:45:50,771 ERROR org.apache.zookeeper.server.NIOServerCnxn: Unexpected Exception:
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:153)
at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1075)
at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1112)
at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
at org.apache.zookeeper.server.DataTree.deleteNode(DataTree.java:594)
at org.apache.zookeeper.server.DataTree.killSession(DataTree.java:966)
at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:818)
at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:965)
at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
at org.apache.zookeeper.server.quorum.Leader$ToBeAppliedRequestProcessor.processRequest(Leader.java:634)
at org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:74)
what should I do to get zookeeper start successful? thanks very much