I am trying to tun the latest version of Kafka (kafka_2.11-0.9.0.0) on windows. I did change the log and tmp directories in all the config files and when I try to start the zookeeper server and getting the below error:
[2015-12-20 18:43:10,826] ERROR Unexpected exception, exiting
abnormally (org.apache.zookeeper.server.ZooKeeperServerMain)
java.io.IOException: Unable to create data directory C:kafka-runtime
mp\version-2
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.(FileTxnSnapLog.java:85)
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:104)
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
C:kafka-runtime is the directory that I have created C:\kafka-runtime, here I can see that "\" is being removed.
Related
I am trying to start up Kafka server locally on a macos with m1 chip. I followed the guide from the official kakfa quickstart(https://kafka.apache.org/quickstart). Zookeeper starts up fine but bin/kafka-server-start.sh config/server.properties is giving me socket invalid argument exception below:
[2023-01-30 09:22:55,790] ERROR Encountered an error while configuring the connection, closing it. (kafka.network.DataPlaneAcceptor)
java.net.SocketException: Invalid argument
at java.base/sun.nio.ch.Net.setIntOption0(Native Method)
at java.base/sun.nio.ch.Net.setSocketOption(Net.java:373)
at java.base/sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:234)
at java.base/sun.nio.ch.SocketAdaptor.setBooleanOption(SocketAdaptor.java:270)
at java.base/sun.nio.ch.SocketAdaptor.setTcpNoDelay(SocketAdaptor.java:305)
at kafka.network.Acceptor.configureAcceptedSocketChannel(SocketServer.scala:759)
at kafka.network.Acceptor.accept(SocketServer.scala:737)
at kafka.network.Acceptor.acceptNewConnections(SocketServer.scala:703)
at kafka.network.Acceptor.run(SocketServer.scala:645)
at java.base/java.lang.Thread.run(Thread.java:829)
I have tried:
Double checking that no other application is using the same port
Use a different JDK (from openjdk17 to openjdk 11 and back to 17)
Rebooted my machine
Clear up kafka related log folder under /tmp
Rebooted my machine
Used a lower version (3.2.1) of kafka tarball (since that one worked for me before but now it also runs into the same socket issue)
Change zookeeper port from 2181 to something else
Turns out to be an antivirus issue. Sorry for the false alarm.
while doing setup facing the following issue
C:\ProgramData\confluent-6.2.1\bin\windows>zookeeper-server-start.bat zookeeper.properties
[2021-10-12 18:16:00,983] INFO Reading configuration from: zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2021-10-12 18:16:00,985] WARN zookeeper.properties is relative. Prepend .\ to indicate that you're sure! (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2021-10-12 18:16:00,986] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing zookeeper.properties
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:157)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:113)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)
Caused by: java.lang.IllegalArgumentException: zookeeper.properties file is missing
at org.apache.zookeeper.server.util.VerifyingFileFactory.doFailForNonExistingPath(VerifyingFileFactory.java:51)
at org.apache.zookeeper.server.util.VerifyingFileFactory.validate(VerifyingFileFactory.java:45)
at org.apache.zookeeper.server.util.VerifyingFileFactory.create(VerifyingFileFactory.java:40)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:142)
... 2 more
Invalid config, exiting abnormally
Does anybody know how to solve this?
It seems that you did not configure the location of the properties file (zookeeper.properties file is missing) in a correct manner. You can fist double check if it exists and then supply the file at launch as in following example from Linux command line (and for a service it will be the same):
bin/zookeeper-server-start.sh -daemon config/zookeeper.properties
Here the folder is relative to the current one, but an absolute path is safer. Beware that in windows the slashes will be inverted, so in your case, the correct command should be something like:
C:\ProgramData\confluent-6.2.1\bin\windows>zookeeper-server-start.bat C:\ProgramData\confluent-6.2.1\config\zookeeper.properties
I am unable to start zookeeper in my local
getting exception
Error: Could not find or load main class software\kafka_2.12-2.3.0\libs\activation-1.1.1.jar;
I have got the same error.
Finally I found the reason.
I put kafka in a folder which it's name contains space like this
C:\Open source\kafka\kafka_2.12-2.3.0
Then I copied all kafka_2.12-2.3.0 to a new folder without space in name, such asc C:\kafka_2.12-2.3.0
and kafka can run
NOte that the command line is: zookeeper-server-start.bat ../../config/zookeeper.properties
NOT: zookeeper-server-start.bat config/zookeeper.properties
Here's my result
I'm getting below error while starting the Kafka-Server on Windows machine. I've downloaded Scala 2.11 - kafka_2.11-2.1.0.tgz from the link: https://kafka.apache.org/downloads and I did the following steps:
Go to config folder in Apache Kafka (C:\Apache-Kafka\kafka_2.11-2.1.0\config) and edit “server.properties” using any text editor.
Find log.dirs and repelace after “=/tmp/kafka-logs” to C:\Apache-Kafka\kafka_2.11-2.1.0\kafka-logs.
Now simply start the server:
>kafka-server-start.bat C:\Apache-Kafka\kafka_2.11-2.1.0\config
Error:
C:\Apache-Kafka\kafka_2.11-2.1.0\bin\windows>kafka-server-start.bat C:\Apache-Kafka\kafka_2.11-2.1.0\config
[2018-12-14 21:09:34,566] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2018-12-14 21:09:34,583] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.nio.file.AccessDeniedException: C:\Apache-Kafka\kafka_2.11-2.1.0\config
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:230)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:560)
at kafka.Kafka$.getPropsFromArgs(Kafka.scala:42)
at kafka.Kafka$.main(Kafka.scala:58)
at kafka.Kafka.main(Kafka.scala)
C:\Apache-Kafka\kafka_2.11-2.1.0\bin\windows>
Note: I've already setup Apache Zookeeper on my Windows machine and it's running on port 2181.
I run the command using run as administrator.
Try this after kafka-server-start.bat
use this: ....\config\server.properties with slash between 2 dots
in my case was
In general we must not use C: drive to store kafka-logs. You can try using the drive other than the C: for storing Kafka logs. It must work.
Change the property log.dirs={Drive other than C:}/tmp/kafka-logs present in KafkaHome/config/server.properties.
I am working through the SSL example which means starting the locator using the properties files instead of default. So it's
start locator --name=IsLocator --properties-file=..\config\gemfire.properties --security-properties-file=..\config\gfsecurity.properties
Then whether the SSL is set in the properties files or not, this causes the locator process to terminate with an error
The Locator process terminated unexpectedly with exit status 1. Please refer to the log file in D:\Geode\bin\IsLocator for full details.
Sep 21, 2017 11:23:22 AM org.apache.geode.distributed.LocatorLauncher failOnStart
INFO: locator is exiting due to an exception
java.lang.RuntimeException: java.io.IOException: Unable to write to deploy directory: D:\tmp
at org.apache.geode.internal.JarDeployer.loadPreviouslyDeployedJarsFromDisk(JarDeployer.java:410)
at org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1191)
at org.apache.geode.internal.cache.GemFireCacheImpl.basicCreate(GemFireCacheImpl.java:758)
at org.apache.geode.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:745)
at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:173)
at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:212)
at org.apache.geode.distributed.internal.InternalLocator.startCache(InternalLocator.java:723)
at org.apache.geode.distributed.internal.InternalLocator.startDistributedSystem(InternalLocator.java:710)
at org.apache.geode.distributed.internal.InternalLocator.startLocator(InternalLocator.java:324)
at org.apache.geode.distributed.LocatorLauncher.start(LocatorLauncher.java:651)
at org.apache.geode.distributed.LocatorLauncher.run(LocatorLauncher.java:553)
at org.apache.geode.distributed.LocatorLauncher.main(LocatorLauncher.java:195)
Caused by: java.io.IOException: Unable to write to deploy directory: D:\tmp
at org.apache.geode.internal.JarDeployer.verifyWritableDeployDirectory(JarDeployer.java:333)
at org.apache.geode.internal.JarDeployer.loadPreviouslyDeployedJarsFromDisk(JarDeployer.java:389)
... 11 more`
The same error happens if GFSH is started as an administrator.
This problem resolves if you remove the deploy-working-dir property.