I am trying to set up Kafka in my local mac - apache-kafka

I have unzipped the kafka_2.12-2.5.0 version and started zookeeper and when I am trying to start kafka using the command
"bin/zookeeper-server-start.sh config/zookeeper.properties" i am getting the following eror:
/Users/manig/Desktop/kafka_2.12-2.5.0/bin/kafka-run-class.sh: line 317: /Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home#/bin/java: No such file or directory
/Users/manig/Desktop/kafka_2.12-2.5.0/bin/kafka-run-class.sh: line 317: exec: /Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home#/bin/java: cannot execute: No such file or directory.
I tried adding double quotes around the JAVA term like this: "$JAVA" and also changing the classpath to cygpath -wp $CLASSPATH but still getting the same above error
can someone please help me with this..!

Try to set your Java ClassPath properly using the below commands. It was not able to recognize you JDK.
export JAVA_HOME="$(/usr/libexec/java_home)"
or
export JAVA_HOME=`/usr/libexec/java_home`

cygpath is for Cygwin, not Mac.
If you want to install on a Mac, just use brew install kafka
Or use Docker

Related

How to fix diesel_cli link libpq.lib error with Postgres tools installed in Docker?

I'm trying (for hours now) to install the cargo crate diesel_cli for postgres. However, every time I run the recommended cargo command:
cargo install diesel_cli --no-default-features --features postgres
I wait a few minutes just to see the same build fail with this message:
note: LINK : fatal error LNK1181: cannot open input file 'libpq.lib'
error: aborting due to previous error
error: failed to compile `diesel_cli v1.4.1`, intermediate artifacts can be found at `C:\Users\<user name here>\AppData\Local\Temp\cargo-installUU2DtT`
Caused by:
could not compile `diesel_cli`.
I'm running postgres in a docker container and have the binaries on my C:\pgsql with the lib and bin directories both on the PATH so I can't figure out why it's not linking. What else could be required they didn't mention in the docs?
In my case the installation was successful but when I tried to run it this error occured.
maybe this would work for others who have the same problem:
open PowerShell
type in setx PQ_LIB_DIR "C:\Program Files\PostgreSQL\13\lib" (or any other path to your PostgreSQL lib)
restart your PC
run again
I had the same issue with WSL, if you're on Linux probably you could find PostgreSQL lib location and add it to your environment variables.
Update:
The answer below is a work around for older versions. Please check the possibility to execute cargo clean first
Original Version
Adding the folder to the PATH variable didn't help, at least in my case, as by some reason it is not used in the /LIBPATH parameter passed to link.exe.
In my case it was C:\Users\<username>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib
You can see it in the beginning of the error message.
Copy libpq.lib in there and it will be used from there.
After installation diesel would require some other assemblies. Copy libcrypto-1_1-x64.dll, libiconv-2.dll and libssl-1_1-x64.dll into the folder showed after where diesel command execution
I had the same error on Ubuntu and for me the following install fixed the issue:
sudo apt install libpq-dev
No need to move files around, just add C:\Program Files\PostgreSQL\14\lib and C:\Program Files\PostgreSQL\14\bin to your PATH. Installing and running diesel should have no problems.
Note: your paths may be different, and remember to close/reopen your terminal so the PATH variable is updated.
(Tested on Windows 10)
To give clear steps for windows:
Add C:\Users<username>.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib in the path in environment variables
Copy libpq.lib that is in C:\Program Files\PostgreSQL\14\lib (obviously this is with version 14) and paste it in C:\Users<username>.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib
If you've attempted to cargo build (or anything that runs the build scripts for libpq rust crate) when your environment was invalid, then you need to do a cargo clean after fixing your environment otherwise you'll still get the libpq.lib not found error even when it's in your PATH. The other answers where you copy the file into another directory are just hacks
You can instead add the path to .../lib to the compilers' library search paths, using RUSTFLAGS environment variable. It works both for installing diesel_cli and for building your projects.
RUSTFLAGS='-L /usr/local/pgsql/lib' cargo build
On Windows with EDB installer, the path contains a space, so use CARGO_ENCODED_RUSTFLAGS instead. For PowerShell:
$env:CARGO_ENCODED_RUSTFLAGS = "-L`u{1f}C:\Program Files\PostgreSQL\14\lib"

Kafka Error: Could not find or load main class config.zookeeper.properties

Searched everywhere, could not find a solution to my problem.
When I go to the directory after unpacking the binary version of Kafka (unpack it via terminal using the command tar -xzf), I try to start Zookeeper with the command (macOS Sierra):
bin/zookeeper-server-start.sh config/zookeeper.properties
I get the same error:
Error: Could not find or load main class config.zookeeper.properties
Download the latest binary version of Kafka (also tried earlier versions) - kafka_2.11-0.11.0.0.tgz - the result is the same - the same error.
In posts here write that the error in the downloaded version - source instead of binary. I only have binary.
Java version with the latest update (build 1.8.0_141-b15).
What could be the problem?
I encountered the same issue on my mac. It turned out the space of the path causing the problem. By removing it, it works.
/Users/xxx/My Projects/kafka_2.11-0.11.0.0 => /Users/xxx/MyProjects/kafka_2.11-0.11.0.0
Please run following commands:
wget http://www-eu.apache.org/dist/kafka/0.11.0.1/kafka_2.11-0.11.0.1.tgz
tar -xzf kafka_2.11-0.11.0.1.tgz
cd kafka_2.11-0.11.0.1
bin/zookeeper-server-start.sh config/zookeeper.properties
Try to use complete path like this:
bin/zookeeper-server-start.sh /usr/local/Cellar/kafka/1.1.0/libexec/config/zookeeper.properties

Sipp with pcap installation on windows

I am trying to install sipp with pcap-replay on winows7.
I have installed cygwnin, libncurse, and winpcap. I was trying to patch cygwin with IPv6 using the URL http://cygwin.win6.jp/cygwin-ipv6/, but it couldn't get the setup.ini file.
So I copied to local directory and upgrade, but it said no new updates.
And if I try to complie sipp. it gives error
$ ./configure.ac --with-pcap
./configure.ac: line 3: syntax error near unexpected token [SIPp],'
./configure.ac: line 3:AC_INIT([SIPp], [3.3], [sipp-users#lists.sourceforge.net], [sipp])'
Could anyone help me in getting sipp be installed on windows?
I download sipp.3.3.990 it has ./configure and was able to install on Linux.
But now on windows it still fails saying ncurse is not present. But I have downloaded it.
How do I troubleshoot it from here?
You can follow below steps to generate configure from configure.ac.
$> autoreconf -i
Now, you will see a configure file created at the same location.
To run the above command, following packages are required: automake,ncurses-devel,libncurses,M4 and autoconf.
if any of the above packages are missing run the cygwin setup file again and select the packages on the package selection screen and click install.

Error from Zookeper 3.4.6 version

I downloaded zookeeper-3.4.6.tar.gz and while executing zkServer.sh start , I am getting below error.I did google but couldn't find a solution, Please let me kno if you see similar issues.
CLASSPATH=/home/spanda20/zookeeper/bin/../src/java/lib/*.jar:/home/spanda20/zookeeper/bin/../conf:
zkServer.sh: 81: /home/spanda20/zookeeper/bin/zkEnv.sh: Syntax error: "(" unexpected (expecting "fi")
Finally I am able to see storm UI and its running.
Below changes I did in the .bashrc file. Please make sure that You add bin path in the PATH VARIABLE.
I did add the below variables in the .bashrc file.
export JAVA_HOME =JDK Path
export ZOOKEPER_HOME =Zookeper install path
Then add them in the variable path.
PATH= $PATH:$JAVA_HOME/bin:$ZOOKEPER_HOME/bin
Thanks
Sanjeeb
I'm using zookeeper version 3.4.9 on ubuntu 14.04 and installed openjdk 7. I have added below lines to ~/.bashrc
export JAVA_HOME="/usr/lib/jvm/java-7-openjdk-amd64"
export ZOOKEEPER_HOME ="/path/to/zookeeper"
PATH=$PATH:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin.
First I ran sh /path-to-zookeeper/bin/zkServer.sh start but didn't not work I got the same error.
I ran it with sudo /bin/bash /path-to-zookeeper/bin/zkServer.sh start it worked.
./zkServer.sh start
This should work.
Edit /etc/environment, and paste the following:
JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64/jre"
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:$JAVA_HOME/bin"
modify the JAVA_HOME as per your local Java version.

openjdk-1.7.0_55 on ubuntu: Could not load the property file 'output_xml.properties' for output method 'xml'

On openjdk-1.7.0_55 on ubuntu I am getting exception:
com.sun.org.apache.xml.internal.serializer.utils.WrappedRuntimeException: Could not load the property file 'output_xml.properties' for output method 'xml'
Can this be classpath issue?
I can find "output_xml.properties" in /usr/lib/jvm/java-1.7.0-openjdk-amd64/jre/lib/resources.jar.
Any help is appreciated.
Thanks!
The issue might have been caused by a recent Java update and incomplete services restart per this page.
you need to include the xalan JARs in the ../jre/lib/endorsed folder
from http://xml.apache.org/xalan-j/faq.html#jdk14:
Place the xalan.jar, serializer.jar, xercesImpl.jar and xml-apis.jar
in the \lib\endorsed directory, where is where
the runtime software is installed.
Try ensuring that your system is consistently using the same Java version - sometimes these get out of sync.
First do echo $JAVA_HOME and verify that it points to /usr/lib/jvm/java-1.7.0-openjdk-amd64/.
Now run sudo update-alternatives --config java and select the correct number for /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
Now do the same for javac: sudo update-alternatives --config javac.
If you're using SYS V init scripts and running in a container such a Tomcat, DON'T start the script directly with:
sudo /etc/init.d/tomcat7 start
That will start it without a clean environment.
Instead, use the service command:
sudo /sbin/service tomcat7 start