I would like to use Mirror Maker 2 for our data pipeline and take advantage of its features in prod but I don't find anything related to whether its officially release or still beta version.
MirrorMaker 2 was released as part of Apache Kafka 2.4.0. You can verify this in the release notes, and see the specific JIRA here.
Related
we have production Kafka cluster with 2.7 version , 5 Kafka brokers on RHEL 7.9 version
we want to upgrade the Kafka version to 3.X version
3.X version not include zookeeper , so we are wondering if we can do upgrade without any data loss
regarding to kafka 2.7 version , Kafka storing the metadata on zookeeper servers ( as brokers ids , topics names etc )
but is it possible to perform rolling upgrade from 2.7 to 3.x version without any data loss?
The upgrade guide should contain all infos you need.
While KRaft mode (without ZooKeeper) is production ready since 3.3, they still keep ZooKeeper around for compatibility until the 4.0 release.
Furthermore If I understand correctly, it is currently only possible to set up a fresh cluster in KRaft mode, but not to migrate an existing one with ZooKeeper. Kafka 3.5 will be a migration version they intend you to migrate from ZooKeeper to KRaft.
This is explained quite nicely in the release notes from Kafka, especially for Kafka 3.3 and the release video
As long as your Kafka brokers are not running with Java 8 still, you can simply do a rolling upgrade from 2.7 to 3.X like you are used to.
What kind of benefits do you think upgrading the kafka version will provide and is it important? What are the advantages of switching from 2.8 to 3.1?
The Kafka release notes cover all added features and fixes. For example, as of Kafka 3.3.1 release, Zookeeper is no longer required.
Kafka 3.0 removed support for Scala 2.11, and maybe Java 7/8, and enables transactions by default, I believe.
Make sure that you read the upgrade notes for protocol changes that will make it impossible to downgrade in the event of errors.
Can anyone confirm the KRaft (aka KIP-500) production mode available? I have checked the below links it says the kraft is still an experimental feature and should not be used in production.
Use Kafka for warehousing without Zookeeper
https://adityasridhar.com/posts/how-to-easily-install-kafka-without-zookeeper
The wiki seems like the best place to get status on this: https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready
The kafka team seems to have planned to mark it ready in 3.3, which was supposed to be in August 2022, but it is now September and 3.2.3 was released on Sept 19.
So it sounds like it will be soon, but not just yet.
According to 3.3.1 release notes, it is production ready
https://kafka.apache.org/downloads
I wanted to get started with kafka ( i had little experience with kafka 0.10 , and now, it seems there are "major" changes happened)
I am going through release changes of kafk. However, I am stuck with which version I should choose in terms of stability , community support . I am little reluctant to use the latest version 2.5
Please suggest me a version which didn't get a lot of "major" changes . I heard kafka 2.1 is good start as it had some breaking changes from previous version .
If you're making the leap from 0.10, I don't see any point in stopping part-way with 2.1.
In terms of releases, the latest stable release is 2.5.0. If you prefer, 2.4 was released in December 2019 and has had a dot release since (2.4.1).
Check out this excellent talk which addresses why you should upgrade.
I didn't find a lot of material about 2.5
The majority of material is version agnostic (unless it's based on super-old versions, like 0.10 etc). If you want to see breaking changes specifically, look at the download page which notes major changes and links to the release notes for each version which lists every single change.
I have to upgrade Kafka from 0.10.0 to 2.0.0 for a production system. Is it a stable release? Also, is there a concept of GA, RC, and M2 for Kafka releases?
Yes Kafka 2.0.0 is a stable release and it can be used in production.
Apache Kafka also uses Release Candidates before releasing new stable versions. For example 2.0.0 went through 3 RCs before reaching GA. (RC0, RC1, RC2).
Apache Kafka however does not use Milestone Releases. A Timed-based release plan was voted by the community and is used to produce regular GA (production ready) releases.