How to start Zookeper - apache-kafka

I am unable to start zookpeer to further start Kafka.
My folder structure is:
Root
bin
zookeper-server-start.sh
config
zookeper.properties
I have tried running
./zookeeper-server-start.sh ../config/zookeeper.properties
And i get the following error:
/d/Programs/Kafka/kafka_2.12-2.2.0/bin/kafka-run-class.sh: line 306: C:\Program: No such file or directory
Code executed that generates error
exec $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$#"
When running java version i get:
> java version "1.8.0_211" Java(TM) SE Runtime Environment (build
> 1.8.0_211-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)

Looks like a classpath issue:
Make sure that Java is defined correctly.
Check what exactly happens in kafka-run-class.sh line 306, the chances are that you'll see something like this:
exec $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$#"
So this is a Java invocation line. If you've installed java in "Program Files" (a folder with a space) you'll need to enclose the Env Variable that points to java with double quotes.
check JAVA_HOME env variable
check PATH env evariable

Related

kdb q - oracle access with babel fails

I have installed babel (interface between kdb and other dbs such as oracle) from here. After copying the oracle8 driver to the babel/lib folder I run the command from the babel documentation page
java -Xmx1024m -Doracle.jdbc.defaultRowPrefetch=10000
-cp "babel.jar:lib/ojdbc8.jar" de.skelton.babel.Babel 6868 oracle.jdbc.OracleDriver
which gives me
$ java -Xmx1024m -Doracle.jdbc.defaultRowPrefetch=10000 -cp "babel.jar:ojdbc8.jar" de.skelton.babel.Babel 6868 oracle.jdbc.OracleDriver
Error: Could not find or load main class de.skelton.babel.Babel
I tried to run the alternative
java -Xmx1024m -jar babel.jar 6868 oracle.jdbc.OracleDriver
which gives
Babel for kdb+ v1.34 2014.03.24
Error: could not load jdbc driver. Exiting.
I am running this on Windows 7 and have the following jre installed
java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b31)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b31, mixed mode)
For windows, perhaps you need to use a semi-colon as separator in the classpath arg? e.g.
java -Xmx1024m -Doracle.jdbc.defaultRowPrefetch=10000 -cp "babel.jar;lib/ojdbc8.jar" de.skelton.babel.Babel 6868 oracle.jdbc.OracleDriver

SBT command to check what memory options are on use

After modifying the sbtconfig.txt file I would like to know if there is any command in SBT that would allow me to check if this values are really in effect.
After typing "settings -V" I see lots of options but none of them seem helpful:
Try sbt -v (lowercase v)
I got this on linux:
$ sbt -v master
[process_args] java_version = '1.8.0_112'
# Executing command line:
java
-Dfile.encoding=UTF8
-Xms1G
-Xmx6G
-XX:MaxPermSize=512M
-XX:ReservedCodeCacheSize=250M
-XX:+TieredCompilation
-XX:-UseGCOverheadLimit
-XX:+CMSClassUnloadingEnabled
-XX:+UseConcMarkSweepGC
-Dsbt.override.build.repos=true
-jar
/usr/share/sbt/bin/sbt-launch.jar

Jython ignores jythonpath

My jython script is using Apache Tika, and it fails to load the Tika jar even though I placed it both in CLASSPATH and in PYTHONPATH. Here's what I'm doing:
> export JYTHONPATH=/full_path_to/tika-app-1.5.jar
> export CLASSPATH=/full_path_to/tika-app-1.5.jar
> java -Xmx512m -Xss1024k -jar /full_path_to/jython-standalone-2.7-b2.jar \
-c 'from org.apache.tika import *'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named apache
I'm using Jython 2.7 beta 2 on OSX. Java version is 1.8:
> java -version
java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)
The same thing is happening on Amazon's EC2 Linux with Java 1.7.
When I'm using the non-standalone version of the jython jar (jython-2.7-b2.jar instead of jython-standalone-2.7-b2.jar), it seems to be working okay.
Any idea what's happening?
I think I found the problem. In Java, when using the -jar option, it more or less ignores the classpath. Just have to run the same code but with calling the jython 'main' class directly - not via '-jar'.

Which version of Java does SBT use?

How to find out, which version of JDK SBT uses?
My laptop has both JDK 1.6 and JDK 1.7 installed, so I was just wondering.
You can use eval at the sbt prompt with the appropriate system properties:
> eval System.getProperty("java.version")
[info] ans: String = 1.7.0_45
> eval System.getProperty("java.home")
[info] ans: String = /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
(The other answers are fine too, this is just another method.)
Just run sbt console and you'll get something like this:
Welcome to Scala version 2.10.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_45).
Type in expressions to have them evaluated.
Type :help for more information.
Which shows you the version of Java being used.
Cheers
JC
It's actually easy.
Find the command line of SBT (cat `which sbt`) and see which Java it is.
For example, my SBT command line begins with:
/usr/bin/java -Xmx512M
And /usr/bin/java -version tells the Java version, in my case it is 1.7.
sbt -v is another way:
# sbt -v
[process_args] java_version = '1.7.0_121'
# Executing command line:
java
-Xms1024m
-Xmx1024m
-XX:ReservedCodeCacheSize=128m
-XX:MaxPermSize=256m
-jar
/usr/share/sbt-launcher-packaging/bin/sbt-launch.jar
[info] Loading project definition from /root/.sbt/0.13/staging/a6d8e5030b69785a9763/build/project
[info] Set current project to xx (in build file:/build/)
>
Here's a batch file method I created to get that info immediately from the command line (Windows only, but you can do something similar in a shell script):
#echo off
setlocal
sbt "eval System.getProperty(\"java.version\")" "eval System.getProperty(\"java.home\")"
#echo on

scala: Error occurred during initialization of VM on Ubuntu 12.04

I tried to work with easy example of scala language.
I installed sbt successfully. When run sbt -h I see help message from sbt:
Usage: sbt [options]
-h | -help print this message
-v | -verbose this runner is chattier
-d | -debug set sbt log level to debug
-no-colors disable ANSI color codes
-sbt-create start sbt even if current directory contains no sbt project
-sbt-dir <path> path to global settings/plugins directory (default: ~/.sbt)
-sbt-boot <path> path to shared boot directory (default: ~/.sbt/boot in 0.11 series)
-ivy <path> path to local Ivy repository (default: ~/.ivy2)
-mem <integer> set memory options (default: 1536, which is -Xms1536m -Xmx1536m -XX:MaxPermSize=384m -XX:ReservedCodeCacheSize=192m) ....
but when I tried to run in the directory with scala assignments (it's Coursera course) I caught error:
nazar_art#nazar-desctop:~/scala/example$ sbt
Detected sbt version 0.12.1
Starting sbt: invoke with -help for other options
Error occurred during initialization of VM
Could not reserve enough space for object heap
but it should open the sbt command prompt. Smt like this:
shell$ cd /path/to/progfun-project-directory # This is the shell of the operating system
shell$ sbt
> _ # This is the sbt shell
UPDATE:
After I add this following line
java -Xms64M -Xmx256M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=64M -jardirname $0/sbt-launch.jar "$#"
at /usr/bin I see next output:
nazar_art#nazar-desctop:/usr/bin$ sbt
bash: /usr/bin/sbt: Permission denied
nazar_art#nazar-desctop:/usr/bin$ sudo sbt
sudo: sbt: command not found
- Why this happen?
- How to solve this issue?
First of all, The example folder name (and the folder name that you have installed sbt) cannot include "!". this probably applies to other special characters as well. Even, I recommend to install SBT in a folder that has no space on its path (and the same thing for examples folder).
For example, if you have installed SBT (or examples folder) on a path like:
/Users/dashti/My Programs/sbt-0.12.1
You can re-install it a path like:
/Users/dashti/MyPrograms/sbt-0.12.1
By applying this naming change, if the error is gone, please ignore the rest of this post.
Otherwise, to fix the mentioned error, first find sbt executable script, then edit it and correct any problem in it:
If you are using a UNIX-based operating system, please follow these steps:
1- Find SBT: Open a terminal and type this command:
which sbt
and you will receive a path like:
/Users/dashti/Documents/scala-2.10.0/bin/sbt
and in this file, there is a command for executing a java program, like:
java -Xms512M -Xmx1024M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=128M -jar `dirname $0`/sbt-launch.jar "$#"
2- Edit sbt script: by entering this command in terminal:
vim /Users/dashti/Documents/scala-2.10.0/bin/sbt
you will be able to edit this file and maybe you should change the memory usage arguments to a lower specification ( You should think about whether you have enough RAM to be running the sbt). For example, this one will be a better choice:
java -Xms64M -Xmx256M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=64M -jar `dirname $0`/sbt-launch.jar "$#"
SBT needs more memory allocated to the JVM.
This can be done in build.sbt:
javaOptions ++= Seq(
"-Xms64m",
"-Xmx256M",
"-XX:+CMSClassUnloadingEnabled",
"-XX:MaxPermSize=64M"
);
Creating an sbt file(no extentions) and running ./sbt is better, but with the code below(#MohammadDashti's suggestion), sbt-launch.jar needs to be in the project folder.
java -Xms512M -Xmx1024M -Xss1M -XX:+CMSClassUnloadingEnabled \
-XX:MaxPermSize=128M -jar `dirname $0`/sbt-launch.jar "$#"
Later you could add javaagent:/path/to/jrebel -Drebel.remoting_plugin=true and re-deploy changes on save with a free JRebel for Scala license.