submit task to Spark - scala

Question

I installed spark on ubuntu 14.04 following this tutorial http://blog.prabeeshk.com/blog/2014/10/31/install-apache-spark-on-ubuntu-14-dot-04/
I am able to run the examples provided inside spark and it seems to work.
The problem is that I am not able to make a scala file and to execute it with spark. This is what I have done following the guidelines https://spark.apache.org/docs/latest/quick-start.html
My standalone app is:
/* SimpleApp.scala */
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import org.apache.commons.math3.random.RandomDataGenerator
object SimpleApp {
def main(args: Array[String]) {
val logFile = "/home/donbeo/Applications/spark/spark-1.1.0/README.md" // Should be some file on your system
val conf = new SparkConf().setAppName("Simple Application")
val sc = new SparkContext(conf)
val logData = sc.textFile(logFile, 2).cache()
val numAs = logData.filter(line => line.contains("a")).count()
val numBs = logData.filter(line => line.contains("b")).count()
println("Lines with a: %s, Lines with b: %s".format(numAs, numBs))
println("A random number")
val randomData = new RandomDataGenerator()
println(randomData.nextLong(0, 100))
}
}
and my sbt file is :
name := "Simple Project"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"
libraryDependencies += "org.apache.commons" % "commons-math3" % "3.3"
my project structure is:
donbeo#donbeo-HP-EliteBook-Folio-9470m:~/Documents/scala_code/simpleApp$ find .
.
./src
./src/main
./src/main/scala
./src/main/scala/SimpleApp.scala~
./src/main/scala/SimpleApp.scala
./simple.sbt
donbeo#donbeo-HP-EliteBook-Folio-9470m:~/Documents/scala_code/simpleApp$
and then I run
donbeo#donbeo-HP-EliteBook-Folio-9470m:~/Documents/scala_code/simpleApp$ sbt package
[info] Set current project to Simple Project (in build file:/home/donbeo/Documents/scala_code/simpleApp/)
[info] Updating {file:/home/donbeo/Documents/scala_code/simpleApp/}simpleapp...
[info] Resolving org.eclipse.jetty.orbit#javax.transaction;1.1.1.v201105210645 .[info] Resolving org.eclipse.jetty.orbit#javax.mail.glassfish;1.4.1.v20100508202[info] Resolving org.eclipse.jetty.orbit#javax.activation;1.1.0.v201105071233 ..[info] Resolving org.spark-project.akka#akka-remote_2.10;2.2.3-shaded-protobuf .[info] Resolving org.spark-project.akka#akka-actor_2.10;2.2.3-shaded-protobuf ..[info] Resolving org.spark-project.akka#akka-slf4j_2.10;2.2.3-shaded-protobuf ..[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /home/donbeo/Documents/scala_code/simpleApp/target/scala-2.10/classes...
[info] Packaging /home/donbeo/Documents/scala_code/simpleApp/target/scala-2.10/simple-project_2.10-1.0.jar ...
[info] Done packaging.
[success] Total time: 8 s, completed 04-Feb-2015 15:20:09
donbeo#donbeo-HP-EliteBook-Folio-9470m:~/Documents/scala_code/simpleApp$
and at the final step I get an error
donbeo#donbeo-HP-EliteBook-Folio-9470m:~/Applications/spark/spark-1.1.0$ ./bin/spark-submit \ --class "SimpleApp" \ --master local[4] \ /home/donbeo/Documents/scala_code/simpleApp/target/scala-2.10/simple-project_2.10-1.0.jar
Exception in thread "main" java.net.URISyntaxException: Illegal character in path at index 0: --class
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.parseHierarchical(URI.java:3086)
at java.net.URI$Parser.parse(URI.java:3044)
at java.net.URI.<init>(URI.java:595)
at org.apache.spark.util.Utils$.resolveURI(Utils.scala:1343)
at org.apache.spark.deploy.SparkSubmitArguments.parse$1(SparkSubmitArguments.scala:338)
at org.apache.spark.deploy.SparkSubmitArguments.parseOpts(SparkSubmitArguments.scala:225)
at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:60)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:70)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
donbeo#donbeo-HP-EliteBook-Folio-9470m:~/Applications/spark/spark-1.1.0$
Am I doing something wrong? How can I solve?

Answer 1

You need to remove all the \ from their command line examples, they have been added because of the line breaks:
./bin/spark-submit --class "SimpleApp" --master local[4] /home/donbeo/Documents/scala_code/simpleApp/target/scala-2.10/simple-project_2.10-1.0.jar