I am used to view the status of Zookeeper in the Solr web admin interface. After upgrading to ZK 3.5.5 from 3.4.13 the Solr web interface are showing an error.
Is there something wrong with my setup or does solr admin 7.7 not work with ZK 3.5?
According to the Apache Solr Reference Guide 7.7 - Setting Up an External ZooKeeper Ensemble:
Solr currently uses Apache ZooKeeper v3.4.13.
And according to SOLR-8346 Upgrade Zookeeper to version 3.5.5 you'd have to upgrade to Solr 8.2 or 9 to use any ZooKeeper 3.5.x version.
Related
we have production Kafka cluster with 2.7 version , 5 Kafka brokers on RHEL 7.9 version
we want to upgrade the Kafka version to 3.X version
3.X version not include zookeeper , so we are wondering if we can do upgrade without any data loss
regarding to kafka 2.7 version , Kafka storing the metadata on zookeeper servers ( as brokers ids , topics names etc )
but is it possible to perform rolling upgrade from 2.7 to 3.x version without any data loss?
The upgrade guide should contain all infos you need.
While KRaft mode (without ZooKeeper) is production ready since 3.3, they still keep ZooKeeper around for compatibility until the 4.0 release.
Furthermore If I understand correctly, it is currently only possible to set up a fresh cluster in KRaft mode, but not to migrate an existing one with ZooKeeper. Kafka 3.5 will be a migration version they intend you to migrate from ZooKeeper to KRaft.
This is explained quite nicely in the release notes from Kafka, especially for Kafka 3.3 and the release video
As long as your Kafka brokers are not running with Java 8 still, you can simply do a rolling upgrade from 2.7 to 3.X like you are used to.
We have installed zookeeper 3.6.2 and Kafka 2.13-2.6.0
Recently I noticed that zookeeper-server-start.sh is actually using the zookeeper jar file from /kafka/libs/ which is zookeeper-3.5.8.jar.
How do I upgrade zookeeper to 3.6.2. Do I have to find a version of kafka that has it bundled in the tar?
Why do they ask you to download and install zookeeper if the jar is already bundled in kafka and that is the one that kafka uses?
You will need to separately install (and start) a Zookeeper cluster outside of the versions and scripts provided by Kafka if you want to upgrade it, but there are risks involved.
Kafka does come bundled with Zookeeper and is tested against specific versions, but it is not upgraded as frequently as Zookeeper project does releases. You should follow the Kafka upgrade notes before blindly upgrading dependency components; there are some notes around version 2.6 that mentions situations where Zookeeper upgrades can fail and what settings need to be added to fix it. And unless you can guarantee a Zookeeper server upgrade will be backwards compatible, Kafka still has a version-specific Zookeeper client for network communications.
Worth pointing out that Kafka KRaft (Zookeeper-less mode) is already in preview release phase with newest 3.x releases, and is scheduled to be closer to production-ready by the end of the year, so I would hold off on upgrading Zookeeper unless you actually need if for something else.
I am building a Hortonworks cluster with both HDP and HDF to be installed. I first installed HDP and then installed/integrated with HDF on top of it.
Environment Details:
OS: Red Hat Enterprise Linux Server release 7.6 (Maipo)
Version: Ambari -2.7.3, HDP - 3.1, HDF -3.4.0
Basically HDP-3.1 has kafka 1.0.1 in the package and in HDF has kafka 2.1.0 packages are available and I need HDF version of Kafka to be available. Though I had installed Kafka from HDF, Ambari shows the kafka version of 1.0.1. After integration with HDF, it's not showing up the Kafka-2.1.0 in the Add service list.
I need to know, how can I get Kafka 2.1.0 installed in the cluster.
Also, Is it a possibility that version showed is 1.0.1, though Kafka 2.1.0 is installed.
Is it a possibility that version showed is 1.0.1, though Kafka 2.1.0 is installed
Doubtful. Ambari parses the packages that are installed on the machine to determine the versions
My suggestion would be to manually SSH to each machine and try to install Kafka from yum and see what versions are available.
If you appropriately setup the HDF YUM repos, then that version of Kafka should be available.
Alternatively, you could always install Kafka/Zookeeper externally and manage it outside of Ambari
I have installed CDH 5.16 Express using packages in a RHEL server. I am trying to install Kafka now and i observed that it can be installed only if CDH is installed as parcels.
1) Is it possible to install Kafka or confluent platform" separately in the server and use it along withCDH` components.
2) Is there any other workaround to install Kafka using Cloudera Manager
In order use the CDK 4.0 (cloudera distribution of Kafka) with Cloudera 5.13, I was forced to install CDK 4.0 as a parcel.
I had a cloudera quickstart docker VM that I downloaded. It runs without Kerberos authentication. After starting the quickstart VM, I separately installed the quickstart Kafka from Apache kafka's website. This was required as the kafka packaged within cloudera was a older version. Since, this was non kerberos environment, the Kafka server upon startup started using the zookeeper that was running in quickstart VM. This way I achieved connection of Kafka with cloudera VM.
If you are new to CDH/CM then I suggest you first try and use the Kafka service that is bundled within Cloudera. Go to 'Add Service' within Cloudera drop-down and select kafka. Enabling this Kafka service will give you a set of brokers for kafka to run. Also, Kafka needs Zookeeper to run. Zookeeper comes by default in Cloudera. So, you would get a working cluster with kafka enabled in it. You can think of changing to the latest version of Kafka (using the approach mentioned above) once you are comfortable with inbuilt tools of CDH/CM.
I install Storm and Ambari UI in a Ubuntu machine.
But now I want to join the Storm with ambari UI. Is there any tutorial? anyone have tips?
Note: I have just installed on the virtual machine the Storm, Kafka and Ambari server (default).
I know that there is a VM NortonWorks with these pre-installed services, but the idea is to install on a virgin machine.
thanks :)
Are you trying to manage and monitor Storm through Ambari? If so then you must provision Storm through Ambari. You can do this by logging into the ambari UI then clicking the actions buttons and select 'Add Service'. Follow the service installation wizard to install Storm. During this process you will be able to configure Storm to your needs. Storm is available in HDP versions 2.1 and higher.