I am just starting out with ScalaTest and following http://www.scalatest.org/quick_start page
I followed the steps as outlined (by downloading the jar file linked above the code)
and the code fails
unittest vi ExampleSpec.scala
➜ unittest scalac -cp scalatest-app_2.13-3.0.8.jar ExampleSpec.scala
➜ unittest scala -cp scalatest-app_2.13-3.0.8.jar org.scalatest.run ExampleSpec
An exception or error caused a run to abort. This may have been caused by a problematic custom reporter.
java.lang.NoClassDefFoundError: scala/xml/NamespaceBinding
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1368)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1033)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1011)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1509)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1011)
at org.scalatest.tools.Runner$.main(Runner.scala:827)
at org.scalatest.run$.main(run.scala:120)
at org.scalatest.run.main(run.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at scala.reflect.internal.util.ScalaClassLoader.$anonfun$run$2(ScalaClassLoader.scala:105)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:40)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:37)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:130)
at scala.reflect.internal.util.ScalaClassLoader.run(ScalaClassLoader.scala:105)
at scala.reflect.internal.util.ScalaClassLoader.run$(ScalaClassLoader.scala:97)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:130)
at scala.tools.nsc.CommonRunner.run(ObjectRunner.scala:29)
at scala.tools.nsc.CommonRunner.run$(ObjectRunner.scala:28)
at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:43)
at scala.tools.nsc.CommonRunner.runAndCatch(ObjectRunner.scala:35)
at scala.tools.nsc.CommonRunner.runAndCatch$(ObjectRunner.scala:34)
at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:70)
at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:91)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:103)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:108)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
Caused by: java.lang.ClassNotFoundException: scala.xml.NamespaceBinding
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 29 more
My Scala Version is
➜ unittest scala -version
Scala code runner version 2.13.1 -- Copyright 2002-2019, LAMP/EPFL and Lightbend, Inc.
I thought the error is because scala-xml is a separate project than Scala, so I downloaded the jar file from https://mvnrepository.com/artifact/org.scala-lang/scala-xml/2.11.0-M4
Next, I tried the entire process again and it fails with different error
unittest scala -classpath "scalatest-app_2.13-3.0.8.jar:scala-xml-2.11.0-M4.jar" org.scalatest.run ExampleSpec
An exception or error caused a run to abort. This may have been caused by a problematic custom reporter.
java.lang.NoClassDefFoundError: scala/Serializable
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:515)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:423)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:417)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:691)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:416)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1368)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1033)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1011)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1509)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1011)
at org.scalatest.tools.Runner$.main(Runner.scala:827)
at org.scalatest.run$.main(run.scala:120)
at org.scalatest.run.main(run.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at scala.reflect.internal.util.ScalaClassLoader.$anonfun$run$2(ScalaClassLoader.scala:105)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:40)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:37)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:130)
at scala.reflect.internal.util.ScalaClassLoader.run(ScalaClassLoader.scala:105)
at scala.reflect.internal.util.ScalaClassLoader.run$(ScalaClassLoader.scala:97)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:130)
at scala.tools.nsc.CommonRunner.run(ObjectRunner.scala:29)
at scala.tools.nsc.CommonRunner.run$(ObjectRunner.scala:28)
at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:43)
at scala.tools.nsc.CommonRunner.runAndCatch(ObjectRunner.scala:35)
at scala.tools.nsc.CommonRunner.runAndCatch$(ObjectRunner.scala:34)
at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:70)
at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:91)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:103)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:108)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
Caused by: java.lang.ClassNotFoundException: scala.Serializable
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 39 more
Could someone please help me understand what the issue is?
Thank you
UPDATE (SOLVED)
As per the comment from #J0HN and answer from #Mario, I was able to solve the issue
➜ unittest ll
total 17608
-rw-r--r-- 1 harit staff 504B Nov 21 17:15 ExampleSpec.scala
-rw-r--r--# 1 harit staff 555K Nov 22 10:35 scala-xml_2.13-1.2.0.jar
-rw-r--r--# 1 harit staff 7.9M Nov 21 16:59 scalatest-app_2.13-3.0.8.jar
➜ unittest scalac -classpath "scalatest-app_2.13-3.0.8.jar:scala-xml_2.13-1.2.0.jar" ExampleSpec.scala
➜ unittest scala -classpath "scalatest-app_2.13-3.0.8.jar:scala-xml_2.13-1.2.0.jar" org.scalatest.run ExampleSpec
Run starting. Expected test count is: 2
ExampleSpec:
A Stack
- should pop values in last-in-first-out order
- should throw NoSuchElementException if an empty stack is popped
Run completed in 506 milliseconds.
Total number of tests run: 2
Suites: completed 1, aborted 0
Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
➜ unittest
I have downloaded scala-xml_2.13-1.2.0.jar from this page
Scala 2.13 decoupled scala-xml from the standard library:
The following modules are no longer included in the distribution:
scala-xml, scala-parser-combinators, scala-swing. They are
community-maintained and published to Maven Central.
As suggested by J0HN, download scala-xml_2.13 and try the following command
scala -cp scalatest-app_2.13-3.0.8.jar:scala-xml_2.13-1.2.0.jar org.scalatest.run ExampleSpec
Related
I'm trying to Dockerize my (otherwise running nice) very small akka-http webapp. I'm using the buildfile indicated here. For convenience added:
FROM alpine:latest as packager
RUN apk --no-cache add openjdk11-jdk openjdk11-jmods
ENV JAVA_MINIMAL="/opt/java-minimal"
# build minimal JRE
RUN /usr/lib/jvm/java-11-openjdk/bin/jlink \
--verbose \
--add-modules \
java.base,java.sql,java.naming,java.desktop,java.management,java.security.jgss,java.instrument \
--compress 2 --strip-debug --no-header-files --no-man-pages \
--release-info="add:IMPLEMENTOR=radistao:IMPLEMENTOR_VERSION=radistao_JRE" \
--output "$JAVA_MINIMAL"
FROM alpine:latest
ENV JAVA_HOME=/opt/java-minimal
ENV PATH="$PATH:$JAVA_HOME/bin"
COPY --from=packager "$JAVA_HOME" "$JAVA_HOME"
COPY build/libs/application.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
However it fails to start with error below. Because I'm using a minimalistic setup for jre11 I think I might be missing some modules. But I'm not sure, and I don't know which java modules are mandatory for an akka-http scala-12 application. Hopefully somebody knows??
Exception in thread "main" java.lang.ExceptionInInitializerError
at akka.dispatch.AbstractNodeQueue.<clinit>(AbstractNodeQueue.java:181)
at akka.actor.LightArrayRevolverScheduler.<init>(LightArrayRevolverScheduler.scala:157)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
at akka.actor.ReflectiveDynamicAccess.$anonfun$createInstanceFor$1(ReflectiveDynamicAccess.scala:33)
at scala.util.Try$.apply(Try.scala:209)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(ReflectiveDynamicAccess.scala:28)
at akka.actor.ReflectiveDynamicAccess.$anonfun$createInstanceFor$5(ReflectiveDynamicAccess.scala:39)
at scala.util.Success.flatMap(Try.scala:247)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(ReflectiveDynamicAccess.scala:39)
at akka.actor.ActorSystemImpl.createScheduler(ActorSystem.scala:925)
at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:786)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:246)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:289)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:234)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:225)
at nl.d8r.stretchsketch.WebServer$.main(Server.scala:37)
at nl.d8r.stretchsketch.WebServer.main(Server.scala)
Caused by: java.lang.ExceptionInInitializerError
at akka.util.Unsafe.<clinit>(Unsafe.java:42)
at akka.dispatch.AbstractNodeQueue.<clinit>(AbstractNodeQueue.java:179)
... 19 more
Caused by: java.lang.NoClassDefFoundError: sun/misc/Unsafe
at akka.util.Unsafe.<clinit>(Unsafe.java:22)
... 20 more
Caused by: java.lang.ClassNotFoundException: sun.misc.Unsafe
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
... 21 more
You need to add a jvm argument to your ENTRYPOINT clause:
--add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED
and should look like the following:
ENTRYPOINT ["java", "--add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED", "-jar","/app.jar"]
Here's a brilliant web talk by J. A. De Goes: https://www.youtube.com/watch?v=sxudIMiOo68 - highly recommended for everyone interested in functional programming
And here's accompanying code gist:
https://gist.github.com/jdegoes/1b43f43e2d1e845201de853815ab3cb9
When I run $ scalac fpmax.scala, it compiles everything to a new directory fpmax
But then when I run scala App0, it gives me error:
Exception in thread "main" java.lang.NoClassDefFoundError: App0 (wrong name: fpmax/App0)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at scala.reflect.internal.util.ScalaClassLoader.$anonfun$tryClass$1(ScalaClassLoader.scala:45)
at scala.util.control.Exception$Catch.$anonfun$opt$1(Exception.scala:242)
at scala.util.control.Exception$Catch.apply(Exception.scala:224)
at scala.util.control.Exception$Catch.opt(Exception.scala:242)
at scala.reflect.internal.util.ScalaClassLoader.tryClass(ScalaClassLoader.scala:45)
at scala.reflect.internal.util.ScalaClassLoader.tryToLoadClass(ScalaClassLoader.scala:39)
at scala.reflect.internal.util.ScalaClassLoader.tryToLoadClass$(ScalaClassLoader.scala:39)
at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.tryToLoadClass(ScalaClassLoader.scala:125)
at scala.reflect.internal.util.ScalaClassLoader$.classExists(ScalaClassLoader.scala:150)
at scala.tools.nsc.GenericRunnerCommand.guessHowToRun(GenericRunnerCommand.scala:36)
at scala.tools.nsc.GenericRunnerCommand.<init>(GenericRunnerCommand.scala:55)
at scala.tools.nsc.GenericRunnerCommand.<init>(GenericRunnerCommand.scala:18)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:42)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:101)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
I run scala 2.12.6.
What would be correct way to run this code snippet?
I had to edit it a bit to run App0.
Change def main: Unit to def main(args: Array[String]): Unit, so that it has main method with args as per Scala spec.
Let's compile it:
scala fpmax.scala. Indeed, this will create fpmax directory with all classes in it
Then run the App0:
scala fpmax.App0 5.50 ✔ 12:21 19.03.19
What is your name?
Alex
Hello, Alex, welcome to the game!
Dear Alex, please guess a number from 1 to 5:
4
You guessed wrong, Alex! The number was: 3
Do you want to continue, Alex?
n
Note, I am NOT going inside the fpmax directory with classes
I am trying to submit a job to spark on my machine as so:
$ spark-submit --master local --class ai.affable.flint.Foo target/scala-2.11/flint.jar
However, this fails with the following error:
java.lang.ClassNotFoundException: ai.affable.flint.Foo
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
I have verfied that the JAR file exists and has a class called Foo:
$ jar tvf ./target/scala-2.11/flint.jar | grep Foo
2003 Fri Dec 14 20:53:40 MYT 2018 ai/affable/flint/Foo.class
...
This baffles me because:
a) the JAR exists b) the class exists in the jar 3) I have specified the fully qualified path and double checked for any path errors or mispellings.
Does anyone know what I am missing?
EDIT:
I got it to work by recreating the project in a fresh directory.I literally copy pasted the code and repeated the steps.
I will still like to know what I can do in situations like this short of recreating the project.
I am trying to run all the tests (all of them extend FlatSpec) in a jar using the ScalaTest Runner, but get a message for VerifyError. However, I am able to individually run the tests.
The jar was compiled using sbt test:assembly. I am trying to run the tests in another environment where sbt is not available. The tests are available in the jar in com/tfs/test path, as shown in the output below:
samik#samik-lap:~/git/proj$ jar tf test-2018.2.jar | grep MyTest
com/tfs/test/MyTest$$anonfun$1$$anonfun$apply$mcV$sp$1.class
com/tfs/test/MyTest$$anonfun$4.class
com/tfs/test/MyTest$$anonfun$5.class
com/tfs/test/MyTest$$typecreator4$1.class
com/tfs/test/MyTest$$typecreator5$1.class
com/tfs/test/MyTest$$typecreator9$1.class
com/tfs/test/MyTest.class
The following command runs the specific test just fine:
samik#samik-lap:~/git/proj$ scala -J-Xmx2g -cp "scalatest_2.11-3.0.5.jar:scalactic_2.11-3.0.5.jar:test-2018.2.jar" org.scalatest.run com.tfs.test.MyTest
0 [ScalaTest-main] INFO com.tfs.test.MyTest - Starting MyTest test application
Start Time: 0 sec
...
However, when I use Runner to run all the tests (there are ~7 similar tests in the jar available at the same path), it doesn't work.
samik#samik-lap:~/git/proj$ scala -J-Xmx2g -cp "scalatest_2.11-3.0.5.jar:scalactic_2.11-3.0.5.jar" org.scalatest.tools.Runner -o -R test-2018.2.jar
Discovery starting.
*** RUN ABORTED ***
java.lang.VerifyError: Cannot inherit from final class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
How do I get all the tests to run through Runner? How do I get to know what is happening inside and where the issue is? Thanks for any pointer.
I'm trying to run the word count example from the Quick start page
import com.nicta.scoobi.Scoobi._
import Reduction._
object WordCount extends ScoobiApp {
def run() {
val lines = fromTextFile(args(0))
val counts = lines.mapFlatten(_.split(" "))
.map(word => (word, 1))
.groupByKey
.combine(Sum.int)
counts.toTextFile(args(1)).persist
}
}
It works fine when I use in memory mode, but when trying local mode (or cluster mode) I fail with the errors:
[WARN] LocalJobRunner - job_local_0001 <java.lang.RuntimeException: java.lang.ClassNotFoundException: Class scala.runtime.Nothing$ not found>java.lang.RuntimeException: java.lang.ClassNotFoundException: Class scala.runtime.Nothing$ not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1439)
at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat.com$nicta$scoobi$impl$mapreducer$ChannelOutputFormat$$mkTaskContext$1(ChannelOutputFormat.scala:63)
at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat$$anonfun$getContext$1.apply(ChannelOutputFormat.scala:75)
at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat$$anonfun$getContext$1.apply(ChannelOutputFormat.scala:75)
at scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:189)
at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:91)
at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat.getContext(ChannelOutputFormat.scala:75)
at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat.write(ChannelOutputFormat.scala:43)
at com.nicta.scoobi.impl.plan.mscr.MscrOutputChannel$$anon$5$$anonfun$write$1.apply(OutputChannel.scala:137)
at com.nicta.scoobi.impl.plan.mscr.MscrOutputChannel$$anon$5$$anonfun$write$1.apply(OutputChannel.scala:135)
at scala.collection.immutable.List.foreach(List.scala:318)
at com.nicta.scoobi.impl.plan.mscr.MscrOutputChannel$$anon$5.write(OutputChannel.scala:135)
at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel$$anonfun$reduce$1.apply$mcV$sp(OutputChannel.scala:201)
at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel$$anonfun$reduce$1.apply(OutputChannel.scala:201)
at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel$$anonfun$reduce$1.apply(OutputChannel.scala:201)
at scala.Option.getOrElse(Option.scala:120)
at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel.reduce(OutputChannel.scala:200)
at com.nicta.scoobi.impl.mapreducer.MscrReducer$$anonfun$reduce$1.apply(MscrReducer.scala:55)
at com.nicta.scoobi.impl.mapreducer.MscrReducer$$anonfun$reduce$1.apply(MscrReducer.scala:52)
at scala.Option.foreach(Option.scala:236)
at com.nicta.scoobi.impl.mapreducer.MscrReducer.reduce(MscrReducer.scala:52)
at com.nicta.scoobi.impl.mapreducer.MscrReducer.reduce(MscrReducer.scala:33)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:164)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:572)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:309)
Caused by: java.lang.ClassNotFoundException: Class scala.runtime.Nothing$ not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1350)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1437)
... 25 more
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/3757970833182018747_-1642337927_156373685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.combiners-step1 of 1
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/1307074498433974065_910223079_156373685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.mappers-step1 of 1
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-624792843022440048_-470268278_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TG23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-7527273518266336656_-470264434_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TK23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-7162952586058180219_-470259629_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TP23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-1228551315878554095_-470253863_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TV23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/6598684265640022340_1943382592_156373685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.reducers-step1 of 1
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/1699308645513763631_1905624154_156371685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/env/a88809af-334b-499e-bafc-1a2ebeffdfbd
[INFO] MapReduceJob - Map 100% Reduce 0%
[error] (run-main) com.nicta.scoobi.impl.exec.JobExecException: MapReduce job 'job_local_0001' failed! Please see http://localhost:8080/ for more info.
com.nicta.scoobi.impl.exec.JobExecException: MapReduce job 'job_local_0001' failed! Please see http://localhost:8080/ for more info.
at com.nicta.scoobi.impl.exec.MapReduceJob.report(MapReduceJob.scala:80)
at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$reportMscr$1.apply(HadoopMode.scala:157)
at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$reportMscr$1.apply(HadoopMode.scala:154)
at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:54)
at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:53)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at com.nicta.scoobi.impl.exec.HadoopMode$Execution.runMscrs(HadoopMode.scala:133)
at com.nicta.scoobi.impl.exec.HadoopMode$Execution.execute(HadoopMode.scala:115)
at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeLayer$1.apply(HadoopMode.scala:105)
at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeLayer$1.apply(HadoopMode.scala:104)
at org.kiama.attribution.AttributionCore$CachedAttribute.apply(AttributionCore.scala:61)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at com.nicta.scoobi.impl.exec.HadoopMode.com$nicta$scoobi$impl$exec$HadoopMode$$executeLayers$1(HadoopMode.scala:68)
at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeNode$1.apply(HadoopMode.scala:91)
at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeNode$1.apply(HadoopMode.scala:84)
at org.kiama.attribution.AttributionCore$CachedAttribute.apply(AttributionCore.scala:61)
at scalaz.syntax.IdOps$class.$bar$greater(IdOps.scala:15)
at scalaz.syntax.ToIdOps$$anon$1.$bar$greater(IdOps.scala:78)
at com.nicta.scoobi.impl.exec.HadoopMode.execute(HadoopMode.scala:52)
at com.nicta.scoobi.impl.exec.HadoopMode.execute(HadoopMode.scala:48)
at com.nicta.scoobi.impl.Persister.persist(Persister.scala:44)
at com.nicta.scoobi.impl.ScoobiConfigurationImpl.persist(ScoobiConfigurationImpl.scala:355)
at com.nicta.scoobi.application.Persist$class.persist(Persist.scala:33)
at p.WordCount$.persist(scoobi-test.scala:6)
at com.nicta.scoobi.application.Persist$PersistableList.persist(Persist.scala:151)
at p.WordCount$.run(scoobi-test.scala:14)
at com.nicta.scoobi.application.ScoobiApp$$anonfun$main$1.apply$mcV$sp(ScoobiApp.scala:80)
at com.nicta.scoobi.application.ScoobiApp$$anonfun$main$1.apply(ScoobiApp.scala:75)
at com.nicta.scoobi.application.ScoobiApp$$anonfun$main$1.apply(ScoobiApp.scala:75)
at com.nicta.scoobi.application.LocalHadoop$class.runOnLocal(LocalHadoop.scala:41)
at p.WordCount$.runOnLocal(scoobi-test.scala:6)
at com.nicta.scoobi.application.LocalHadoop$class.executeOnLocal(LocalHadoop.scala:35)
at p.WordCount$.executeOnLocal(scoobi-test.scala:6)
at com.nicta.scoobi.application.LocalHadoop$$anonfun$onLocal$1.apply(LocalHadoop.scala:29)
at com.nicta.scoobi.application.InMemoryHadoop$class.withTimer(InMemory.scala:71)
at p.WordCount$.withTimer(scoobi-test.scala:6)
at com.nicta.scoobi.application.InMemoryHadoop$class.showTime(InMemory.scala:79)
at p.WordCount$.showTime(scoobi-test.scala:6)
at com.nicta.scoobi.application.LocalHadoop$class.onLocal(LocalHadoop.scala:29)
at p.WordCount$.onLocal(scoobi-test.scala:6)
at com.nicta.scoobi.application.Hadoop$class.onHadoop(Hadoop.scala:60)
at p.WordCount$.onHadoop(scoobi-test.scala:6)
at com.nicta.scoobi.application.ScoobiApp$class.main(ScoobiApp.scala:75)
at p.WordCount$.main(scoobi-test.scala:6)
at p.WordCount.main(scoobi-test.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
[trace] Stack trace suppressed: run last compile:runMain for the full output.
[INFO] Task - Communication exception: java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at org.apache.hadoop.mapred.Task$TaskReporter.run(Task.java:648)
at java.lang.Thread.run(Thread.java:679)
java.lang.RuntimeException: Nonzero exit code: 1
at scala.sys.package$.error(package.scala:27)
[trace] Stack trace suppressed: run last compile:runMain for the full output.
[error] (compile:runMain) Nonzero exit code: 1
[error] Total time: 9 s, completed Nov 24, 2013 3:55:30 AM
running the very similar example from github (https://github.com/NICTA/scoobi/tree/SCOOBI-0.7.3/examples/wordCount) does work.
any ideas?
EDIT
I ran the sample according to the explanations in scoobi quick start
running the sample is done using sbt commands:
sbt compile
sbt "run-main mypackage.myapp.WordCount input-files output"
There is no reference regarding how or where to supply parameters such as the location of external jars.
Installed scala version and the one specified in build.sbt must be same. Once this is same, scala-library is auto included into classpath.
We've faced the same issue recently and solved it by adding fork := true to the sbt settings.