I'm trying to use confluent CLI for the first time. I've successfully installed it and I can run the confluent command. But when I try to start confluent with the command confluent local services start I have this error:
Obviously there is a bad configuration issue with the path to the zookeeper.properties but I can't find how this path is built. It seems that it takes my CONFLUENT_HOME variable and add "etc/kafka/zookeeper.properties" to it but I don't understand why.
Can someone explain what I'm doing wrong?
Related
I'm trying to run this command : confluent local services start
I don't know why each time it gives me an error before passing to the next step. So I had to run it again over and over until it passes all the steps.
what is the reason for the error and how to solve the problem?
You need to open the log files to inspect any errors that would be happening.
But, it's possible the services are having a race condition. Schema Registry requires Kafka, REST Proxy and Connect require the Schema Registry... Maybe they are not waiting for the previous components to start.
Or maybe your machine does not have enough resources to start all services. E.g. I believe at least 6GB of RAM are necessary. If you have 8GB on the machine, and Chrome and lots of other services are running, for example, then you wouldn't have 6GB readily available.
I am seeing various different issues while trying to start Kafka services on wsl2. Details/symptoms below:
Confluent Kafka (7.0.0) platform
wsl2 - ubuntu 20.04LTS
When I use the command:
confluent local services start
Typically the system will take a long time and then exit with service failed (e.g. zookeeper, as that is the first service to start).
If I check the logs, it is actually started. So I again type the command and sure enough it immediately says zookeeper up, then proceed to try start kafka, which again after a min will say failed to start (but it really has started).
I suspect after starting the service (which is quite fast), system is not able to communicate back/exit and thus times out, I am not sure where the logs related to this are.
Can see this in the screenshot below
This means to start the whole stack (zookeeper/kafka/schema-registry/kafka-rest/kafka-connect/etc), takes forever, and in between I start getting other errors (sometimes, schema-registry is not able to find the cluster id, sometimes its a log file related error), which means I need to destroy and start again.
I have tried this over a couple of days and cant get this to work. Is confluent kafka that unstable on windows or I am missing some config change.
In terms of setup, I have not done any change in the config and am using the default config/ports.
I wanted to install the Apache Kafka but stuck in installing the Zookeeper
I extracted all the file and created the environment variables as well and now it stopped working after
"Zookeeper audit is disabled."
Now after following through these links,
https://www.programmersought.com/article/22066571206/
https://zookeeper.apache.org/doc/r3.6.2/zookeeperAuditLogs.html
I updated both the files (skServer.cmd and conf/zoo.cfg). Adder a line "-Dzookeeper.audit.enable=true" in skServer.cmd file and "audit.enable=true" in conf/zoo.cfg.
Now the output has changed to "Zookeeper audit is Enabled." but still it doesn't do anything and stops there like before.
Output after running the zkServer command with files edited
Even tried changing one file at a time but the same output and still stops working.
Can anyone help me understand the problem and provide a solution as well?
Thank you so much.
zookeeper-server-start or zkServer.cmd do not return, by design. It's tailing the logs and waiting for a client connection
You must start a second terminal to run the Kafka broker, as mentioned in the Kafka documentation
Kill all java services.
In Linux run killall -9 java
Re re-run the zookeeper and Kafka server.
It works
I am following a quick start guide for getting kafka up and running.
I have zookeeper running but when I try to start kafka server with the following command:
.\bin\windows\kafka-server-start.bat .\config\server.properties
I get the following error
\Teradata\Client\16.20\bin\ was unexpected at this time.
I cant even begin to understand how the two might be related.
Any tip is appreciated. I am very much stuck....
In my case I needed to change the CLASSPATH environment variable, changing from Program Files to Progra~1 and Program Files (x86) to Progra~2
This solved the problem.
This isn't specific to Kafka or Zookeeper, but how they load the Java Home and classpath.
You need to fix your PATH variable in Windows settings to not contain any spaces
looks like the kafka process needs all environment variables in the CLASSPATH to be present. I was getting the same error for Meld which I had deleted but CLASSPATH still have a reference to that.
Removing it from the CLASSPATH worked.
If you are using the Apache Kafka package structure as it is, then fix the path.it will be resolved automatically.
downloaded apache Kafka folder structure kafka_2.11-0.9.0.0 ->bin->windows-> All *.bat files.
kafka_2.11-0.9.0.0->config-> all *.properties files
so to start zookeeper go to the kafka_2.11-0.9.0.0 ->bin->windows->in window address bar type cmd and enter after that enter:
zookeeper-server-start.bat ..\..\config/zookeeper.properties
o/p: zookeeper will start working
To start broker again go to the same folder
kafka_2.11-0.9.0.0 ->bin->windows->in window address bar type cmd and enter after that enter:
kafka-server-start.bat ..\..\config/server.properties
o/p: broker will start working.
Note: you can update your own configuration using this properties file.like change in port etc..
I can't even begin to understand how the two might be related.<br/>
For more information, you can check the blog: Role of Apache ZooKeeper in Kafka — Monitoring & Configuration
I am going through guide here:
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-cluster-creation-for-windows-server
Section "Step 1B: Create a multi-machine cluster".
I have installed Cluster on one box and trying to use the same json (as per instructions) and trying to install it on another box so that i can have Cluster running on 2 VMs.
I am now getting this error when I run by TestConfig.ps1:
Previous Fabric installation detected on machine XXX. Please clean the machine.
Previous Fabric installation detected on machine XXX. Please clean the machine.
Data Root node Dev Box1 exists on machine XXX in \XXX\C$\ProgramData\SF\Dev Box1. This is an artifact from a previous installation - please delete the directory corresponding to this node.
First, take a look on this link. These are the requirements for each cluster node that are need to be met if you want to create the cluster.
The error is pretty obvious. You most likely have already SF installed on the machine. So either you have SF runtime or some uncleaned cluster data there.
Your first try should be running CleanFabric powershell script from the SF standalone package on each node. It should clean all SF data (cluster, runtime, registry etc.). Try this and then run TestConfiguration script once again. If this does not help, you would have to go to each node and manually delete any SF data that TestConfiguration script is complaining about.