I am facing issue when starting Kafka server in my local machine (Windows 10) using command bin\windows\kafka-server-start \config\server.properties. I am getting error below. I already have zookeeper server running.
[2018-12-26 12:03:14,124] INFO Registered kafka:type=kafka.Log4jController
MBean (kafka.utils.Log4jControllerRegistration$)
[2018-12-26 12:03:14,155] ERROR Exiting Kafka due to fatal exception
(kafka.Kafka$)
java.nio.file.NoSuchFileException: \config\server.properties
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:230)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:560)
at kafka.Kafka$.getPropsFromArgs(Kafka.scala:42)
at kafka.Kafka$.main(Kafka.scala:58)
at kafka.Kafka.main(Kafka.scala)
I'm currently using kafka 2.12-2.1.1 version.
For me it worked when I initialize the kafka server with the following input:
kafka-server-start.bat \Tools\kafka_2.12-2.1.1\config\server.properties
Obs: Command source from the kafka_2.12-2.1.1\bin\windows
Obs2: I created the Tools folder inside C: to put the kafka and zookeeper files inside.
Running kafka-server-start.bat from within:
C:\Apache\kafka_2.12–2.3.1\bin\windows>
and using a relative path like this:
kafka-server-start.bat ../../config/server.properties
worked for me, for both kafka_2.12–2.3.1 and kafka-2.4.0.
If you are running kafka-server-start from kafka home directory, remove "\" before config. It should do the magic.
bin\windows\kafka-server-start config\server.properties
Thanks,
Naveen
For those who have added kafka binary's path to window's environment PATH variable and still stuck : Passing relative path of server.properties (like ..\..\config\server.properties) with respect to binary location (as added in PATH) will not work. You will have to pass absolute path for server.properties file.
It is not able to find the server.properties file in the same folder.
Provide the absolute path as below and it runs successfully.
kafka-server-start.bat C:\DEVTools\kafka_2.12-2.3.1\config\server.properties
Related
I started zookeeper and kafka from the downloaded package here:
http://mirror.klaus-uwe.me/apache/kafka/2.4.0/kafka_2.13-2.4.0.tgz
Then I tried with this:
https://github.com/confluentinc/schema-registry
1.in one command prompt I start zookeeper after changing the folder in \config\zookeeper.properties - line:
dataDir=C:/KafkaConfl/zookeeper-data (folder created by me)
2.changed also the config of the kafka server in the config\server.properties - line:
- log.dirs=c:\KafkaConfl\kafka-logs
start zookeeper in one cmd prompt:
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
wait and start kafka server in another cmd prompt:
.\bin\windows\kafka-server-start.bat .\config\server.properties
try with the schema registry:
.\bin\windows\schema-registry-start.bat C:\kafkaconfl\etc\schema-registry.properties
Error: Could not find or load main class io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain
Caused by: java.lang.ClassNotFoundException: io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain
Anybody with experience in Kafka / Windows can help, please?
I'm trying to set up a kafka server for a project. I was following this guide - https://kafka.apache.org/quickstart . I ran zookeper successfuly, but when I try to start the kafka server, I get this error bellow. I tried different pathing to the config properties, but I get the same message.
bin\windows\zookeeper-server-start.bat C:\Users\Atanas0104\Documents\kafka_2.12-2.3.0\config\server.properties
[2019-08-15 18:00:20,434] INFO Reading configuration from: C:\Users\Atanas0104\Documents\kafka_2.12-2.3.0\config\server.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
log4j:ERROR Failed to rename [C:\Users\Atanas0104\Documents\kafka_2.12-2.3.0/logs/server.log] to [C:\Users\Atanas0104\Documents\kafka_2.12-2.3.0/logs/server.log.2019-08-15-17].
[2019-08-15 18:00:20,436] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing C:\Users\Atanas0104\Documents\kafka_2.12-2.3.0\config\server.properties
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:156)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:104)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
Caused by: java.lang.NumberFormatException: For input string: "initial.rebalance.delay.ms"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:244)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:152)
... 2 more
Invalid config, exiting abnormally
This is the error message
Mistake is you are running zookeeper-server-start.bat with kafka server .properties,you need to try with kafka-server-start.bat
1) first go to folder where kafka was there and try this
.\bin\windows\kafka-server-start.bat .\config\server.properties
follow the link https://dzone.com/articles/running-apache-kafka-on-windows-os
I think your server.properties is bad, I downloaded it and I went through the same situation, I only changed the last line from
group.initial.rebalance.delay.ms=0
to
initial.rebalance.delay.ms=0
And it did work.
Using the following link I have tried running kafka in windows Setting Up and Running Apache Kafka on Windows OS
I am able to run the zookeeper without any errors and when I try to run the command
.\bin\windows\kafka-server-start.bat .\config\server.properties
I am getting the following error
ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.lang.IllegalArgumentException: Error creating broker listeners
from 'http://localhost:9092': No security protocol defined for
listener HTTP
at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:312)
at kafka.server.KafkaConfig.advertisedListeners(KafkaConfig.scala:1334)
at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1396)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1374)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1063)
at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:1043)
at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
at kafka.Kafka$.main(Kafka.scala:59)
at kafka.Kafka.main(Kafka.scala)
Please direct me in resolving the above issue
Thanks
I was able to resolve the errors stated by doing the following things
I reinstalled Kafka
I made sure that my environmental path for Java is pointing to a 64 but VM only.
I am trying to setup Confluent-4.1.1 on Ubuntu 16.04. To start the ZooKeeper server, I ran ./bin/zookeeper-server-start ./etc/kafka/zookeeper.properties.txt from the root directory of Confluent by following this tutorial.
The error that comes up is-
log4j:ERROR Could not read configuration file from URL [file:./bin/../config/log4j.properties].
java.io.FileNotFoundException: ./bin/../config/log4j.properties (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at java.io.FileInputStream.<init>(FileInputStream.java:101)
at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:557)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.<clinit>(QuorumPeerMain.java:64)
log4j:ERROR Ignoring configuration file [file:./bin/../config/log4j.properties].
log4j:WARN No appenders could be found for logger (org.apache.zookeeper.server.quorum.QuorumPeerConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
I am new to kafka, and I have no clue what this means. Any help in resolving this would be appreciated.
The link you're following is only Apache Kafka, not Confluent, though they should work similarly at least for starting Zookeeper.
If you've downloaded the Confluent distribution, though, and want a single node cluster, you can use the Confluent CLI
To start Zookeeper, Kafka, and the rest of the Confluent Platform, run
./bin/confluent start
Otherwise, the Zookeeper startup script doesn't use a txt file, and it might be unable to detect where you've extracted the tarball, so instead you can use apt like a normal software package
https://docs.confluent.io/current/installation/installing_cp/deb-ubuntu.html
According to the documentation in the link
1:run these commands after changing path-to-confluent with your path
export CONFLUENT_HOME=
export PATH="${CONFLUENT_HOME}/bin:$PATH"
(these commands will make the "confluent" command recognizable from
terminal)
2: run below command
confluent local services start
(this command will start all the services including zookeeper ,
kafka, schema registry etc)
When I start Confluent, Schema-registry fails, preventing the process from completing successfully. This is the response I get:
Starting zookeeper
zookeeper is [UP]
Starting kafka
kafka is [UP]
Starting schema-registry
Schema Registry failed to start
schema-registry is [DOWN]
Starting kafka-rest
Kafka Rest failed to start
kafka-rest is [DOWN]
Starting connect
connect is [UP]
When I tried to run the processes individually, zookeeper ran without problems. However, when I launched kafka, zookeeper displayed the following error:
Error Path:/brokers Error:KeeperErrorCode = NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
Then, when I attempted to run Schema registry, I was hit with a massive list of errors. I'm sure the errors all point to one small thing. Here are some of the errors (many repeat in the same long message):
1.
WARNING: HK2 service reification failed for [org.glassfish.jersey.message.internal.DataSourceProvider] with an exception:
MultiException stack 1 of 2
java.lang.NoClassDefFoundError: javax/activation/DataSource
2.
MultiException stack 2 of 2
java.lang.IllegalArgumentException: Errors were discovered while reifying SystemDescriptor
3.
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of org.glassfish.jersey.server.validation.internal.ValidationBinder$ConfiguredValidatorProvider errors were found
4.
java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
Some of the errors vary slightly based on location, but for the most part, these 4 errors are printed out dozens of times.
I did my best to make sure no ports were being used by other processes. I also stopped and destroyed all instances of confluent that I've created before. I've played around with Kafka on this computer before, so I theorize that that could have something to do with it, but I've made sure to close all past zookeeper and kafka instances.
I've tried to run confluent on a different computer and didn't run into any issues. Does anyone know what could be the problem? I can send the entire error message and provide any additional details.
Thanks in advance!
Remove Java 9.
I had both Java 9 and Java 8 on my computer. Turns out, Confluent was attempting to use Java 9, which isn't compatible with Confluent. When I deleted everything related to Java 9, Confluent started using Java 8, which solved the problem.
As BluePhantom pointed out, using Java 7 will also do the trick.