I'm attempting to run a Scala script that imports scala.IO.Source, which throws an exception every time I try to run it as follows:
Exception in thread "main" java.lang.RuntimeException: Cannot figure out how to run target: countchars1.scala
at scala.sys.package$.error(package.scala:27)
at scala.tools.nsc.GenericRunnerCommand.scala$tools$nsc$GenericRunnerCommand$$guessHowToRun(GenericRunnerCommand.scala:38)
at scala.tools.nsc.GenericRunnerCommand$$anonfun$2.apply(GenericRunnerCommand.scala:48)
at scala.tools.nsc.GenericRunnerCommand$$anonfun$2.apply(GenericRunnerCommand.scala:48)
at scala.Option.getOrElse(Option.scala:108)
at scala.tools.nsc.GenericRunnerCommand.<init>(GenericRunnerCommand.scala:48)
at scala.tools.nsc.GenericRunnerCommand.<init>(GenericRunnerCommand.scala:17)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:33)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:89)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
I believe it is due to my Path or Classpath settings but I'm unsure what these need to be or where they need to be set. I have tried amending .bash_profile to no avail and have tried adding .profile with what I believe to be the required PATH settings but there doesn't appear to be a straight answer to this that I can find
I'm currently running Scala 2.9.1 on a Mac OSX 10.6. The script I'm trying to run from 'Programming in Scala' is as follows:
import scala.io.Source
if (args.length > 0) {
for (line <- Source.fromFile(args(0)).getLines)
print(line.length +" "+ line)
}
else
Console.err.println("Please enter filename")
I encounter the exception when I type the following in the terminal window:
$ scala countchars1.scala countchars1.scala
Any help with this would be much appreciated.
Managed to solve the issue by setting the JAVA_HOME variable using the following instructions:
http://decoding.wordpress.com/2010/02/26/how-to-set-the-java_home-variable-in-mac-os-x-snow-leopard/
Related
I get the below error when I use pyspark via Zeppelin.
The python & spark interpreters work and all environment variables are set correctly.
print os.environ['PYTHONPATH']
/x01/spark_u/spark/python:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python/lib/pyspark.zip:/x01/spark_u/spark/python:/x01/spark_u/spark/python/pyspark:/x01/spark_u/zeppelin/interpreter/python/py4j-0.9.2/src:/x01/spark_u/zeppelin/interpreter/lib/python
zepplin-env.sh is set with the below vars
export PYSPARK_PYTHON=/usr/local/bin/python2
export PYTHONPATH=${SPARK_HOME}/python:${SPARK_HOME}/python/lib/py4j-0.10.4-src.zip:${PYTHONPATH}
export SPARK_YARN_USER_ENV="PYTHONPATH=${PYTHONPATH}"
See the below log file
INFO [2017-11-01 12:30:42,972] ({pool-2-thread-4}
RemoteInterpreter.java[init]:221) - Create remote interpreter
org.apache.zeppelin.spark.PySparkInterpreter
org.apache.zeppelin.interpreter.InterpreterException:
paragraph_1509038605940_-1717438251's Interpreter pyspark not
found
Thank you in advance
I found a workaround for the above issue.The interpreter not found issue does not happen when I create note inside a directory.The issue only happens when I use notes at toplevel.Addionally I foud out that this issue does not happen in 0.7.2 version
Ex :
enter image description here
I followed this quickstart:
https://docs.prediction.io/templates/classification/quickstart/
and this document for evaluation metrics
https://docs.prediction.io/evaluation/paramtuning/
Everything seems ok until the step build and run evaluation metrics
pio eval org.template.classification.AccuracyEvaluation \
org.template.classification.EngineParamsList
I am getting the exception:
Exception in thread "main" scala.reflect.internal.MissingRequirementError: object org.template.classification.AccuracyEvaluation not found.
at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
at scala.reflect.internal.Mirrors$RootsBase.ensureModuleSymbol(Mirrors.scala:126)
at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:161)
at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:21)
at io.prediction.workflow.WorkflowUtils$.getEvaluation(WorkflowUtils.scala:103)
at io.prediction.workflow.CreateWorkflow$$anonfun$19.apply(CreateWorkflow.scala:146)
at io.prediction.workflow.CreateWorkflow$$anonfun$19.apply(CreateWorkflow.scala:144)
Could anyone help me with this?
Thank you very much.
Had the exact same problem. Fixed it by doing the following:
For each .scala file in engine_dir/src/main/scala/org/template/engine_name/ you need to change the first line from...
package <SomeTemplateName>
To the following (replacing engine_name with the name of the folder in the path mentioned above):
package org.template.<engine_name>
Then, in engine.json you need to change the following line...
"engineFactory": "<template name>.<template engine>",
To the following (once again replacing engine_name with the name of the folder in the path mentioned above):
"engineFactory": "org.template.<engine name>.<template engine>",
Now re-run...
pio build
pio train
pio deploy
Then you should be able to run the model evaluation without errors.
Simply run it like this
$ pio eval org.example.classification.AccuracyEvaluation \
org.example.classification.EngineParamsList
You dont have to change anything. The class package from the sample was org.example.classification not org.template.classification
I am a beginner in Spark and trying to follow instructions from here on how to initialize Spark shell from Python using cmd: http://spark.apache.org/docs/latest/quick-start.html
But when I run in cmd the following:
C:\Users\Alex\Desktop\spark-1.4.1-bin-hadoop2.4\>c:\Python27\python bin\pyspark
then I receive the following error message:
File "bin\pyspark", line 21
export SPARK_HOME="$(cd ="$(cd "`dirname "$0"`"/..; pwd)"
SyntaxError: invalid syntax
What am I doing wrong here?
P.S. When in cmd I try just C:\Users\Alex\Desktop\spark-1.4.1-bin-hadoop2.4>bin\pyspark
then I receive ""python" is not recognized as internal or external command, operable program or batch file".
You need to have Python available in the system path, you can add it with setx:
setx path "%path%;C:\Python27"
I'm a fairly new Spark user (as of today, really). I am using spark 1.6.0 on Windows 10 and 7 machines. The following worked for me:
import os
import sys
spark_home = os.environ.get('SPARK_HOME', None)
if not spark_home:
raise ValueError('SPARK_HOME environment variable is not set')
sys.path.insert(0, os.path.join(spark_home, 'python'))
sys.path.insert(0, os.path.join(spark_home, 'C:/spark-1.6.0-bin-hadoop2.6/python/lib/py4j-0.9-src.zip'))
execfile(os.path.join(spark_home, 'python/pyspark/shell.py'))
Using the code above, I was able to launch Spark in an IPython notebook and my Enthought Canopy Python IDE. Before, this, I was only able to launch pyspark through a cmd prompt. The code above will only work if you have your Environment Variables set correctly for Python and Spark (pyspark).
I run these set of path settings whenever I start pyspark in ipython:
import os
import sys
# Sys.setenv('SPARKR_SUBMIT_ARGS'='"--packages" "com.databricks:spark-csv_2.10:1.0.3" "sparkr-shell"') for R
### MANNN restart spart using ipython notebook --profile=pyspark --packages com.databricks:spark-csv_2.10:1.0.3
os.environ['SPARK_HOME']="G:/Spark/spark-1.5.1-bin-hadoop2.6"
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/bin")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python/pyspark/")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python/pyspark/sql")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python/pyspark/mllib")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python/lib")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python/lib/pyspark.zip")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python/lib/py4j-0.8.2.1-src.zip")
sys.path.append("G:/Spark/spark-1.5.1-bin-hadoop2.6/python/lib/pyspark.zip")
from pyspark import SparkContext
from pyspark import SparkConf
from pyspark import SQLContext
##sc.stop() # IF you wish to stop the context
sc = SparkContext("local", "Simple App")
With the reference and help of the user "maxymoo" I was able to find a way to set a PERMANENT path is Windows 7 as well. The instructions are here:
http://geekswithblogs.net/renso/archive/2009/10/21/how-to-set-the-windows-path-in-windows-7.aspx
Simply set path in System -> Environment Variables -> Path
R Path in my system C:\Program Files\R\R-3.2.3\bin
Python Path in my system c:\python27
Spark Path in my system c:\spark-2
The path must be separated by ";" and there must be no space between paths
I want to use Jcurses with Scala on a 64-bit Ubuntu.
Unfortunately i didn't find any tutorial about this subject. Can anybody help me!
My test program "testjcurses.scala"
import jcurses.system._
object TestJcurses {
def main(args:Array[String]) {
println("okay")
Toolkit.init()
}
}
I processed it the following way:
fsc -cp ~/software/Java/jcurses/lib/jcurses.jar:~/software/Java/jcurses/src -d . -Djava.library.path=~/software/Java/jcurses/lib testjcurses.scala
scala -cp ~/software/Java/jcurses/lib/jcurses.jar:~/software/Java/jcurses/src:. -Djava.library.path=~/software/Java/jcurses/lib TestJcurses
The result is:
okay
java.lang.NullPointerException
at jcurses.system.Toolkit.getLibraryPath(Toolkit.java:97)
at jcurses.system.Toolkit.<clinit>(Toolkit.java:37)
at TestJcurses$.main(testjcurses.scala:9)
at TestJcurses.main(testjcurses.scala)
..........
Can anybody help me?
Unfortunately you can't use ~ in bash like that — ~ is expanded to your home dir only right after an (unquoted) space (technically, at the beginning of a bash word, but "after a space" is the simple version). Look how your command line is expanded:
$ echo scala -cp ~/software/Java/jcurses/lib/jcurses.jar:~/software/Java/jcurses/src:. -Djava.library.path=~/software/Java/jcurses/lib TestJcurses
scala -cp /Users/pgiarrusso/software/Java/jcurses/lib/jcurses.jar:~/software/Java/jcurses/src:. -Djava.library.path=~/software/Java/jcurses/lib TestJcurses
As you can see, the ~ is there in the expanded version, and will arrive unchanged to your program, which will be unable to interpret it as anything since tilde expansion is a job for the shell.
Also, you shouldn't need the source directory ~/software/Java/jcurses/src in your classpath (since source files aren't needed to run the program). So try:
scala -cp ~/software/Java/jcurses/lib/jcurses.jar:. -Djava.library.path=$HOME/software/Java/jcurses/lib TestJcurses
I am working on an application that executes a Jython 2.5.3 script from JAVA 1.6.027. The script just open a file using codecs library and it looks like this:
try:
from codecs import open as codecs_open
except ImportError:
print 'ERROR', 'Could not import.'
CODECS_LIST = ['latin-1', 'utf-8', 'utf-16', '1250', '1252']
def open_file(filename, mode):
'''
DOC
'''
for encoding in CODECS_LIST:
try:
f = codecs_open(filename, mode, encoding)
f.read()
f.close()
print 'INFO', "File %s supports encoding %s." % (filename.split("\\")[-1], encoding)
...
except:
...
When I execute this script debugging in Eclipse, everything works OK, but when I execute the part of the JAVA application that invokes this script, I get this error:
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class sun.nio.ch.FileChannelImpl
at java.io.RandomAccessFile.getChannel(RandomAccessFile.java:253)
at org.python.core.io.FileIO.fromRandomAccessFile(FileIO.java:173)
at org.python.core.io.FileIO.<init>(FileIO.java:79)
at org.python.core.io.FileIO.<init>(FileIO.java:57)
at org.python.core.PyFile.<init>(PyFile.java:135)
at org.python.core.PyTraceback.getLine(PyTraceback.java:65)
at org.python.core.PyTraceback.tracebackInfo(PyTraceback.java:38)
at org.python.core.PyTraceback.dumpStack(PyTraceback.java:109)
at org.python.core.PyTraceback.dumpStack(PyTraceback.java:120)
at org.python.core.Py.displayException(Py.java:1080)
at org.python.core.PySystemState.excepthook(PySystemState.java:1242)
at org.python.core.PySystemStateFunctions.__call__(PySystemState.java:1421)
at org.python.core.Py.printException(Py.java:1053)
at org.python.core.Py.printException(Py.java:1012)
at org.python.util.jython.run(jython.java:264)
at org.python.util.jython.main(jython.java:129)
The JAVA application is able to execute others similar jython scripts. I have detected that the class sun.nio.ch.FileChannelImpl is in the library rt.jar, which is inside /bin/common/ folder and included in my classpath via jvm.cfg file:
...
#LIBRARY PATH
./bin/common;...
...
The same way I do it with other libraries that work fine.
I have been stacked with this problem for a few days, so any help will be appreciated.
Thank you
The problem has been resolved by reinstalling Java Runtime Environment, in my case version jre-6u45
This happened to me because the mysql package that I had installed was installed globally and required root privileges, so when running java, I had to include sudo to get it to work correctly.