Does anyone knows how to fix the error when creating a topic in Kafka?
C:\kafka\bin\windows>kafka-topics.bat --create --bootstrap-server localhost:2181 --replication-factor 1 --partition 1 --topic test
Exception in thread "main" joptsimple.UnrecognizedOptionException: partition is not a recognized option
at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
at joptsimple.OptionParser.parse(OptionParser.java:396)
at kafka.admin.TopicCommand$TopicCommandOptions.<init>(TopicCommand.scala:567)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:47)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
The parameter is partitions
The bootstrap server normally (default) runs in port 9092
C:\kafka\bin\windows>kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
In recent versions, you don't have to create topics on zookeeper. You can directly create topics on the bootstrap servers of Kafka. In the later version, they plan to remove the zookeeper altogether, so they are preparing for that in the current versions.
Use the below to create a new partition. I suggest adding the below parameters as well to control the topic behaviour appropriately.
kafka-topics.bat --create --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1 --config retention.ms=604800000 segment.bytes=26214400 retention.bytes=1048576000 min.insync.replicas=1 --topic test
Related
I am new to kafka and zookepper, and I am trying to create a topic, but I am getting this error -
Exception in thread "main" joptsimple.UnrecognizedOptionException: zookeeper is not a recognized option
at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
at joptsimple.OptionParser.parse(OptionParser.java:396)
at kafka.admin.TopicCommand$TopicCommandOptions.<init>(TopicCommand.scala:517)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:47)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
I am using this command to create the topic -
.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partions 1 --topic TestTopic
Newer versions(2.2+) of Kafka no longer requires ZooKeeper connection string
--zookeeper localhost:2181
It throws the following exception while creating a topic
Exception in thread "main" joptsimple.UnrecognizedOptionException:
zookeeper is not a recognized option
Instead, add Kafka Broker --bootstrap-server localhost:9092 connection string.
./kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 4
In the latest version kafka_2.12-3.1.0 (2022) after unzipping and setting the properties and logs. keep the Kafka folder on the C drive and always run the command prompt with 'run as administrator'.
The .bat file is for windows
Terminal 1
C:\kafka\bin\windows>zookeeper-server-start.bat
..\..\config\zookeeper.properties
Terminal 2
C:\kafka\bin\windows>kafka-server-start.bat
..\..\config\server.properties
Terminal 3
C:\kafka\bin\windows>kafka-topics.bat --create --topic tutorialspedia
--bootstrap-server localhost:9092
Created topic tutorialspedia.
To checklist of topic created
C:\kafka\bin\windows>kafka-topics.bat --list --bootstrap-server
localhost:9092
tutorialspedia
Read the official Kafka documentation for the version you downloaded, and not some other blog/article that you might have copied the command from
zookeeper is almost never used for CLI commands in current versions
If you run bin\kafka-topics on its own with --help or no options, then it'll print the help messaging that shows all available arguments.
It's not --partions, but --partitions.
Same like you, see this link: Exception: partition is not a recognized option, when creating a kafka topic inside a docker.
Kakfa create Topic :
./kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 4
Kafka Created Topics for Windows:
kafka-topics.bat --create --topic learningkafka --bootstrap-server localhost:9092
After topic created we can check the kafka list:
kafka-topics.bat --list --bootstrap-server localhost:9092
I cannot create Kafka topic from GCE
in GEC
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
aa#kafka-1-vm:~$ sudo systemctl restart kafka
aa#kafka-1-vm:~$ bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test_cdc
-bash: bin/kafka-topics.sh: No such file or directory
aa#kafka-1-vm:~$ kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --
topic test_cdc
Exception in thread "main" joptsimple.UnrecognizedOptionException: bootstrap-server is not a recognized option
at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
at joptsimple.OptionParser.parse(OptionParser.java:396)
at kafka.admin.TopicCommand$TopicCommandOptions.<init>(TopicCommand.scala:361)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:44)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
If you use version older than 2.2 you should use "--zookeeper" option and pass connection string to zookeeper (check your kafka version).
Command would be something like this:
kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test_cdc
[before your command] kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test_cdc
In the latest version kafka_2.12-3.1.0 (2022) after unzipping and setting the properties and logs. keep the Kafka folder on the C drive and always run the command prompt with 'run as administrator'.
The .bat file is for windows
Terminal 1
C:\kafka\bin\windows>zookeeper-server-start.bat
..\..\config\zookeeper.properties
Terminal 2
C:\kafka\bin\windows>kafka-server-start.bat
..\..\config\server.properties
Terminal 3
C:\kafka\bin\windows>kafka-topics.bat --create --topic tutorialGB
--bootstrap-server localhost:9092
Created topic tutorialGB.
To checklist of topic created
C:\kafka\bin\windows>kafka-topics.bat --list --bootstrap-server
localhost:9092
tutorialGB
For this i started Zookeeper:
bin/zookeeper-server-start.sh config/zookeeper.properties
then the kafka server:
bin/kafka-server-start.sh config/server.properties
topic:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic fast-messages
then what is the next step for file generation ?
You need to parse the file, and write a Kafka producer from it.
Or you can just pipe the file directly into the Console Producer.
How to write a file to Kafka Producer
Or use KafkaConnect FileSourceConnector
For this same Question i also referred another link and i got the result. The Link is --> https://docs.confluent.io/current/streams/quickstart.html
Followings are the step that i followed-->
Step 1: Start Zookeeper
bin/zookeeper-server-start.sh config/zookeeper.properties
Step 2: Start kafka server
bin/kafka-server-start.sh config/server.properties
Step 3: create a topic named as "fast-messages"
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic fast-messages
step 4: Run Producer
kafka-console-producer.sh --broker-list localhost:9092 --topic fast-messages --new-producer < my_file.txt
step 5: Listened by the Consumer:
/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fast-messages --from-beginning
I am new to Kafka and trying to create a new topic on my local machine.
I am following this link.
Here are the steps which i followed:
Start zookeeper
bin/zookeeper-server-start.sh config/zookeeper.properties
Start kafka-server
bin/kafka-server-start.sh config/server.properties
Create a topic
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
but when creating the topic, i am getting the following error:
Exception in thread "main" joptsimple.UnrecognizedOptionException: bootstrap-server is not a recognized option
at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
at joptsimple.OptionParser.parse(OptionParser.java:396)
at kafka.admin.TopicCommand$TopicCommandOptions.<init>(TopicCommand.scala:358)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:44)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
Is there any other configuration required to create a topic?
What wrong am i doing
If you use version older than 2.2 you should use --zookeeper option and pass connection string to zookeeper
Command would be something like that:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
I created the new topic 'rahul' with the following command :
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic rahul
Created topic "rahul".
I also re-checked the topics with
bin/kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
rahhy
rahul`
Now starting the producer:
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic rahul
hey
hi
hello
But when the time comes to consumer to show the messages: there is nothing
As of Kafka 0.9, you don't use Zookeeper for consumption or production
Try kafka-console-consumer --topic rahul --bootstrap-server localhost:9092
There are other ways you can check messages were sent to Kafka - by checking that the offsets of the topic partitions have changed using GetOffsetShell