Why won't my scalatest test compile? (scala.MatchError) - scala

There is all of the code in my project:
package fileSearcher
import org.scalatest.FlatSpec
class FilterCheckerTests extends org.scalatest.FlatSpec {
"Foo" should
"not do terrible things" in {
assert(1 == 1)
}
}
sbt test crashes with scala.MatchError (full details below).
What am I doing wrong?
[info] Compiling 1 Scala source to C:\scala\course\FileSearcher\target\scala-2.1
0\test-classes...
[error]
[error] while compiling: C:\scala\course\FileSearcher\src\test\scala\fileSe
archer\FilterCheckerTests.scala
[error] during phase: typer
[error] library version: version 2.10.4
[error] compiler version: version 2.10.4
[error] reconstructed args: -classpath C:\scala\course\FileSearcher\target\sca
la-2.10\test-classes;C:\scala\course\FileSearcher\target\scala-2.10\classes;C:\U
sers\Max\.ivy2\cache\org.scalatest\scalatest_2.11\bundles\scalatest_2.11-2.2.4.j
ar;C:\Users\Max\.ivy2\cache\org.scala-lang\scala-reflect\jars\scala-reflect-2.11
.2.jar;C:\Users\Max\.ivy2\cache\org.scala-lang.modules\scala-xml_2.11\bundles\sc
ala-xml_2.11-1.0.2.jar;C:\Users\Max\.ivy2\cache\com.novocode\junit-interface\jar
s\junit-interface-0.11.jar;C:\Users\Max\.ivy2\cache\junit\junit\jars\junit-4.11.
jar;C:\Users\Max\.ivy2\cache\org.hamcrest\hamcrest-core\jars\hamcrest-core-1.3.j
ar;C:\Users\Max\.ivy2\cache\org.scala-sbt\test-interface\jars\test-interface-1.0
.jar -bootclasspath C:\Program Files\Java\jdk1.8.0_20\jre\lib\resources.jar;C:\P
rogram Files\Java\jdk1.8.0_20\jre\lib\rt.jar;C:\Program Files\Java\jdk1.8.0_20\j
re\lib\sunrsasign.jar;C:\Program Files\Java\jdk1.8.0_20\jre\lib\jsse.jar;C:\Prog
ram Files\Java\jdk1.8.0_20\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_20\jre
\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_20\jre\lib\jfr.jar;C:\Program F
iles\Java\jdk1.8.0_20\jre\classes;C:\Users\Max\.ivy2\cache\org.scala-lang\scala-
library\jars\scala-library-2.10.4.jar
[error]
[error] last tree to typer: Literal(Constant(true))
[error] symbol: null
[error] symbol definition: null
[error] tpe: Boolean(true)
[error] symbol owners:
[error] context owners: value <local FilterCheckerTests> -> class FilterCh
eckerTests -> package fileSearcher
[error]
[error] == Enclosing template or block ==
[error]
[error] Template( // val <local FilterCheckerTests>: <notype> in class FilterChe
ckerTests
[error] "org.scalatest.FlatSpec" // parents
[error] ValDef(
[error] private
[error] "_"
[error] <tpt>
[error] <empty>
[error] )
[error] // 2 statements
[error] DefDef( // def <init>(): fileSearcher.FilterCheckerTests in class Filt
erCheckerTests
[error] <method>
[error] "<init>"
[error] []
[error] List(Nil)
[error] <tpt> // tree.tpe=fileSearcher.FilterCheckerTests
[error] Block( // tree.tpe=Unit
[error] Apply( // def <init>(): org.scalatest.FlatSpec in class FlatSpec,
tree.tpe=org.scalatest.FlatSpec
[error] FilterCheckerTests.super."<init>" // def <init>(): org.scalatest
.FlatSpec in class FlatSpec, tree.tpe=()org.scalatest.FlatSpec
[error] Nil
[error] )
[error] ()
[error] )
[error] )
[error] Apply(
[error] "Foo".should("not do terrible things")."in"
[error] Apply(
[error] "assert"
[error] Apply( // def ==(x: Int): Boolean in class Int, tree.tpe=Boolean(t
rue)
[error] 1."$eq$eq" // def ==(x: Int): Boolean in class Int, tree.tpe=(x:
Int)Boolean
[error] 1
[error] )
[error] )
[error] )
[error] )
[error]
[error] == Expanded type of tree ==
[error]
[error] ConstantType(value = Constant(true))
[error]
[error] uncaught exception during compilation: scala.MatchError
[trace] Stack trace suppressed: run last test:compile for the full output.
[error] (test:compile) scala.MatchError: false (of class scala.reflect.internal.
Trees$Literal)
[error] Total time: 0 s, completed Jun 20, 2015 11:07:15 AM
1. Waiting for source changes... (press enter to interrupt)

As you can see by looking at the classpath, which is printed by the compiler, you mixed Scala 2.10 with libraries for 2.11. Given that major versions of Scala are binary incompatible, this can never work.
This can be fixed with scalaVersion := "2.11.5" or by setting all dependencies to use the 2.10 versions, which can be done with libraryDependencies += "group" %% "libName" % "version", where %% means that sbt automatically uses the correct library version.

For anyone getting a similar error (as I did) running Scala 2.10 using Maven instead of sbt, the solution would simply be to change the Maven dependency suggested in the ScalaTest website from
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.11</artifactId>
<version>3.0.0</version>
<scope>test</scope>
</dependency>
to
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.10</artifactId>
<version>3.0.0</version>
<scope>test</scope>
</dependency>

Related

java.lang.NoSuchFieldError when using ScalaTest

I'm getting a quite-hard-to-debug error when using ScalaTest. Oddly this seems to occur when my program has no Main object, but not when it does have a Main object. My code is really just using a typeclass with a polymorphic method to get a slice from a list, and looks like this (apologies for the slightly verbose example, I've reduced it down as much as I can):
package sportarray
object ArrayDefs {
abstract class IsArr[A, I0, DT] {
def getElem(self: A, i: Int): DT
def getSlice[R](self: A, r: R)(implicit sliceTc: IsSlice[R]): sliceTc.Out = sliceTc.getSlice(self, r)
trait IsSlice[R] {
type Out
def getSlice(self: A, ref: R): Out
}
object IsSlice {
implicit val iLocTCForInt = new IsSlice[Int] {
type Out = DT
def getSlice(self: A, ref: Int): Out = getElem(self, ref)
}
implicit val iLocTCForList = new IsSlice[List[Int]] {
type Out = List[DT]
def getSlice(self: A, ref: List[Int]): Out = ref.map(getElem(self, _))
}
}
}
object IsArrSyntax {
implicit class IsArrOps[A, I0, T](self: A)(implicit
tc1d: IsArr[A, I0, T]
) {
def getElem(i: Int) = tc1d.getElem(self, i)
def getSlice[R](r: R)(implicit sliceTc: tc1d.IsSlice[R]) = tc1d.getSlice(self, r)
}
}
}
object ListOfListsObj {
import ArrayDefs._
case class List1d[I0, T] (
indices: List[I0],
data: List[T],
)
implicit def list1dIs1dSpArr[A, I0, T] =
new IsArr[List1d[I0, T], I0, T] {
def getElem(self: List1d[I0, T], i: Int) = self.data(i)
}
}
My test is simple and looks like this, in its own file in the test directory:
package sportarray
import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers
class ArraySpec extends AnyFlatSpec with Matchers {
"List" should "act as an array" in {
import ArrayDefs._
import ArrayDefs.IsArrSyntax._
import ListOfListsObj._
val list1d = List1d[Int, Double](List(1,2,3), List(1.0, 2.0, 3.0))
assert(list1d.getSlice(1) == 2.0)
}
}
If I sbt test this code as it is, I get the following error:
java.lang.NoSuchFieldError: sportarray$ArrayDefs$IsArrSyntax$IsArrOps$$tc1d
at sportarray.ArraySpec.$anonfun$new$1(HelloSpec.scala:12)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.flatspec.AnyFlatSpecLike$$anon$5.apply(AnyFlatSpecLike.scala:1683)
I then tried adding a Main class to the program to see if this was a problem with the test, or a problem with the code itself:
object Main extends App {
import ArrayDefs._
import ArrayDefs.IsArrSyntax._
import ListOfListsObj._
val list1d = List1d[Int, Double](List(1,2,3), List(1.0, 2.0, 3.0))
assert(list1d.getSlice(1) == 2.0)
}
This works fine if I sbt run, and, oddly enough, also allows me to run sbt test without any problems. If I then delete this main class, sbt test once again fails. Is anybody able to shed any light on what is going on here?
Thanks for any help!
Actually I ran Scalatest from command line and it succeded
$ scalac -cp .:scalactic_2.13-3.2.2.jar:scalatest_2.13-3.2.2.jar:scalatest-core_2.13-3.2.2.jar:scalatest-flatspec_2.13-3.2.2.jar:scalatest-matchers-core_2.13-3.2.2.jar:scalatest-shouldmatchers_2.13-3.2.2.jar:scalatest-compatible-3.2.2.jar App.scala AppTest.scala
$ scala -cp .:scalactic_2.13-3.2.2.jar:scalatest_2.13-3.2.2.jar:scalatest-core_2.13-3.2.2.jar:scalatest-flatspec_2.13-3.2.2.jar:scalatest-matchers-core_2.13-3.2.2.jar:scalatest-shouldmatchers_2.13-3.2.2.jar:scalatest-compatible-3.2.2.jar:scala-xml_2.13-1.3.0.jar org.scalatest.run AppTest
Run starting. Expected test count is: 1
AppTest:
List
- should act as an array
Run completed in 151 milliseconds.
Total number of tests run: 1
Suites: completed 1, aborted 0
Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
while in IntelliJ and sbt it failed:
(IntelliJ)
An exception or error caused a run to abort: App$ArrayDefs$IsArrSyntax$IsArrOps$$tc1d
java.lang.NoSuchFieldError: App$ArrayDefs$IsArrSyntax$IsArrOps$$tc1d
at AppTest.$anonfun$new$1(AppTest.scala:10)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.flatspec.AnyFlatSpecLike$$anon$5.apply(AnyFlatSpecLike.scala:1683)
at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
at org.scalatest.flatspec.AnyFlatSpec.withFixture(AnyFlatSpec.scala:1685)
at org.scalatest.flatspec.AnyFlatSpecLike.invokeWithFixture$1(AnyFlatSpecLike.scala:1681)
at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTest$1(AnyFlatSpecLike.scala:1693)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.flatspec.AnyFlatSpecLike.runTest(AnyFlatSpecLike.scala:1693)
at org.scalatest.flatspec.AnyFlatSpecLike.runTest$(AnyFlatSpecLike.scala:1675)
at org.scalatest.flatspec.AnyFlatSpec.runTest(AnyFlatSpec.scala:1685)
at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTests$1(AnyFlatSpecLike.scala:1751)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:333)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:390)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:427)
at scala.collection.immutable.List.foreach(List.scala:333)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.flatspec.AnyFlatSpecLike.runTests(AnyFlatSpecLike.scala:1751)
at org.scalatest.flatspec.AnyFlatSpecLike.runTests$(AnyFlatSpecLike.scala:1750)
at org.scalatest.flatspec.AnyFlatSpec.runTests(AnyFlatSpec.scala:1685)
at org.scalatest.Suite.run(Suite.scala:1112)
at org.scalatest.Suite.run$(Suite.scala:1094)
at org.scalatest.flatspec.AnyFlatSpec.org$scalatest$flatspec$AnyFlatSpecLike$$super$run(AnyFlatSpec.scala:1685)
at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$run$1(AnyFlatSpecLike.scala:1796)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.flatspec.AnyFlatSpecLike.run(AnyFlatSpecLike.scala:1796)
at org.scalatest.flatspec.AnyFlatSpecLike.run$(AnyFlatSpecLike.scala:1794)
at org.scalatest.flatspec.AnyFlatSpec.run(AnyFlatSpec.scala:1685)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1320)
at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1314)
at scala.collection.immutable.List.foreach(List.scala:333)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1314)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:993)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:971)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1480)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:971)
at org.scalatest.tools.Runner$.run(Runner.scala:798)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:40)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:27)
(sbt)
[error] java.lang.NoSuchFieldError: App$ArrayDefs$IsArrSyntax$IsArrOps$$tc1d
[error] at AppTest.$anonfun$new$1(AppTest.scala:10)
[error] at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[error] at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[error] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[error] at org.scalatest.Transformer.apply(Transformer.scala:22)
[error] at org.scalatest.Transformer.apply(Transformer.scala:20)
[error] at org.scalatest.flatspec.AnyFlatSpecLike$$anon$5.apply(AnyFlatSpecLike.scala:1683)
[error] at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
[error] at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
[error] at org.scalatest.flatspec.AnyFlatSpec.withFixture(AnyFlatSpec.scala:1685)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.invokeWithFixture$1(AnyFlatSpecLike.scala:1681)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTest$1(AnyFlatSpecLike.scala:1693)
[error] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.runTest(AnyFlatSpecLike.scala:1693)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.runTest$(AnyFlatSpecLike.scala:1675)
[error] at org.scalatest.flatspec.AnyFlatSpec.runTest(AnyFlatSpec.scala:1685)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTests$1(AnyFlatSpecLike.scala:1751)
[error] at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
[error] at scala.collection.immutable.List.foreach(List.scala:333)
[error] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[error] at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:390)
[error] at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:427)
[error] at scala.collection.immutable.List.foreach(List.scala:333)
[error] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[error] at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
[error] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.runTests(AnyFlatSpecLike.scala:1751)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.runTests$(AnyFlatSpecLike.scala:1750)
[error] at org.scalatest.flatspec.AnyFlatSpec.runTests(AnyFlatSpec.scala:1685)
[error] at org.scalatest.Suite.run(Suite.scala:1112)
[error] at org.scalatest.Suite.run$(Suite.scala:1094)
[error] at org.scalatest.flatspec.AnyFlatSpec.org$scalatest$flatspec$AnyFlatSpecLike$$super$run(AnyFlatSpec.scala:1685)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$run$1(AnyFlatSpecLike.scala:1796)
[error] at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.run(AnyFlatSpecLike.scala:1796)
[error] at org.scalatest.flatspec.AnyFlatSpecLike.run$(AnyFlatSpecLike.scala:1794)
[error] at org.scalatest.flatspec.AnyFlatSpec.run(AnyFlatSpec.scala:1685)
[error] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[error] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[error] at sbt.TestRunner.runTest$1(TestFramework.scala:139)
[error] at sbt.TestRunner.run(TestFramework.scala:154)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.$anonfun$apply$1(TestFramework.scala:317)
[error] at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:277)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:317)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:317)
[error] at sbt.TestFunction.apply(TestFramework.scala:329)
[error] at sbt.Tests$.$anonfun$toTask$1(Tests.scala:311)
[error] at sbt.std.Transform$$anon$3.$anonfun$apply$2(Transform.scala:46)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:281)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:19)
[error] at sbt.Execute.work(Execute.scala:290)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:281)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (Test / executeTests) java.lang.NoSuchFieldError: App$ArrayDefs$IsArrSyntax$IsArrOps$$tc1d
So the issue seems to be how IntelliJ and sbt run Scalatest.
NoSuchFieldError Java
Java NoSuchFieldError when using Reflection
https://github.com/unicredit/hbase-rdd/issues/21

Why does maven give me this Scala error doing clean install

I can't seem to find why scala packages are being "thought of" by maven as a member of package org.apache.kafka.streams.scala. I usually use sbt but I have to use maven on this one. Any help would be appreciated.
/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/bin/java -Dmaven.multiModuleProjectDirectory=/Users/jcena1/IdeaProjects/datapipe-scala-merge-transform "-Dmaven.home=/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3" "-Dclassworlds.conf=/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3/bin/m2.conf" "-javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=65172:/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3/boot/plexus-classworlds-2.5.2.jar" org.codehaus.classworlds.Launcher -Didea.version=2018.2.1 install
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for io.confluent:Jointransform:jar:4.1.1
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: org.scala-lang:scala-library:jar -> duplicate declaration of version ${scala.version} # io.confluent:Jointransform:[unknown-version], /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/pom.xml, line 259, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.13
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 13
[INFO] os.detected.classifier: osx-x86_64
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Jointransform 4.1.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) # Jointransform ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) # Jointransform ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.10:add-source (add-source) # Jointransform ---
[INFO] Source directory: /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala added.
[INFO] Source directory: /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/generated-sources added.
[INFO]
[INFO] --- avro-maven-plugin:1.8.2:schema (default) # Jointransform ---
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) # Jointransform ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 28 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) # Jointransform ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 11 source files to /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/classes
[INFO]
[INFO] --- scala-maven-plugin:3.2.1:compile (default) # Jointransform ---
[INFO] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala:-1: info: compiling
[INFO] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/generated-sources:-1: info: compiling
[INFO] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/generated-sources/annotations:-1: info: compiling
[INFO] Compiling 16 source files to /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/classes at 1536749088682
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Filtertransform.scala:10: error: object language is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.language.implicitConversions
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Filtertransform.scala:14: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Filtertransform.scala:75: error: Symbol 'type org.apache.kafka.streams.Consumed' is missing from the classpath.
[ERROR] This symbol is required by 'value com.lightbend.kafka.scala.streams.StreamsBuilderS.consumed'.
[ERROR] Make sure that type Consumed is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[ERROR] A full rebuild may help if 'StreamsBuilderS.class' was compiled against an incompatible version of org.apache.kafka.streams.
[ERROR] val stream: KStreamS[String, Array[Byte]] = builder.stream(inputTopic)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:12: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.JavaConverters._
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:11: error: object language is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.language.implicitConversions
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:20: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:21: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable.ListBuffer
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:39: error: Symbol 'type org.apache.kafka.streams.Consumed' is missing from the classpath.
[ERROR] This symbol is required by 'method com.lightbend.kafka.scala.streams.ImplicitConversions.consumedFromSerde'.
[ERROR] Make sure that type Consumed is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[ERROR] A full rebuild may help if 'ImplicitConversions.class' was compiled against an incompatible version of org.apache.kafka.streams.
[ERROR] (out:+ s)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:57: error: value asScala is not a member of java.util.List[String]
[ERROR] possible cause: maybe a semicolon is missing before `value asScala'?
[ERROR] .asScala
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:147: error: not found: value Consumed
[ERROR] implicit val c = Consumed.`with`(Serdes.String(), Serdes.ByteArray())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:149: error: polymorphic expression cannot be instantiated to expected type;
[ERROR] found : [K, V]org.apache.kafka.streams.kstream.KTable[K,V]
[ERROR] required: org.apache.kafka.streams.scala.kstream.KTable[String,Array[Byte]]
[ERROR] val table: KTable[String, Array[Byte]] = builder.table(inputTopic)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:156: error: not found: value Serialized
[ERROR] implicit val sb = Serialized.`with`(Serdes.String(), Serdes.ByteArray())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:157: error: not found: value Serialized
[ERROR] implicit val ss = Serialized.`with`(Serdes.String(), Serdes.String())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:158: error: not found: value Produced
[ERROR] implicit val p = Produced.`with`(Serdes.String(),Serdes.String())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:241: error: value asScala is not a member of java.util.List[String]
[ERROR] possible cause: maybe a semicolon is missing before `value asScala'?
[ERROR] .asScala
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:245: error: value asScala is not a member of java.util.List[String]
[ERROR] val newValue = getValFromJSONMessage(lv, aggregateColumnList.asScala.toList.head)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:232: error: diverging implicit expansion for type org.apache.kafka.streams.kstream.Serialized[KR,VR]
[ERROR] starting with method consumedFromSerde in object ImplicitConversions
[ERROR] val groupedK = table.groupBy{ (key: String, value: Array[Byte]) =>
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:252: error: not found: type Aggregator
[ERROR] val newAggregator: Aggregator[String, Array[Byte], Array[Byte]] = new Aggregator[String, Array[Byte], Array[Byte]] {
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:252: error: not found: type Aggregator
[ERROR] val newAggregator: Aggregator[String, Array[Byte], Array[Byte]] = new Aggregator[String, Array[Byte], Array[Byte]] {
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:10: error: object language is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.language.implicitConversions
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:11: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.JavaConverters._
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:18: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable.ListBuffer
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:37: error: not found: type ListBuffer
[ERROR] val lst = new ListBuffer[String]
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:84: error: value asScala is not a member of java.util.List[String]
[ERROR] Note: implicit value j is not applicable here because it comes after the application point and it lacks an explicit result type
[ERROR] val leftRekeyColumnList = if(rekeyLeftNeeded) joinStep.getStringList("joinOn.leftFields").asScala.toList else List.empty
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:85: error: value asScala is not a member of java.util.List[String]
[ERROR] Note: implicit value j is not applicable here because it comes after the application point and it lacks an explicit result type
[ERROR] val rightRekeyColumnList = if(rekeyRightNeeded) joinStep.getStringList("joinOn.rightFields").asScala.toList else List.empty
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:96: error: value asScala is not a member of java.util.List[String]
[ERROR] Note: implicit value j is not applicable here because it comes after the application point and it lacks an explicit result type
[ERROR] if(!JoinUtils.validateFilterExpression(filterExpression.asScala.toList)) {
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:147: error: value asScala is not a member of java.util.List[String]
[ERROR] JoinUtils.joinValues(lv, rv, node, outputFields.asScala.toList)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:157: error: value asScala is not a member of java.util.List[String]
[ERROR] val filterPredicate = JoinUtils.buildPostJoinPredicate(filterExpression.asScala.toList, lv)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:165: error: value asScala is not a member of java.util.List[String]
[ERROR] val ba: Array[Byte] = JoinUtils.removeEntryFromJsonNode(extraFields.asScala.toList, value)
[ERROR] ^
[ERROR] 29 errors found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.937 s
[INFO] Finished at: 2018-09-12T06:44:53-04:00
[INFO] Final Memory: 35M/376M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.1:compile (default) on project Jointransform: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Process finished with exit code 1
I needed to prefix _root_ to all the Scala core packages like for example
import _root_.scala.collection.JavaConverters._

Apache Spark Build error

I'm building Apache spark source code in ubuntu 14.04.4 (spark version: 1.6.0 with Scala code runner version 2.10.4) with command
sudo sbt/sbt assembly
and getting the following error,
[warn] def deleteRecursively(dir: TachyonFile, client: TachyonFS) {
[warn] ^
[error] [error] while compiling:
/home/ashish/spark-apps/spark-1.6.1/core/src/main/scala/org/apache/spark/util/random/package.scala
[error] during phase: jvm [error] library
version: version 2.10.5 [error] compiler version: version
2.10.5 [error] reconstructed args: -deprecation -Xplugin:/home/ashish/.ivy2/cache/org.spark-project/genjavadoc-plugin_2.10.5/jars/genjavadoc-plugin_2.10.5-0.9-spark0.jar
-feature -P:genjavadoc:out=/home/ashish/spark-apps/spark-1.6.1/core/target/java -classpath /home/ashish/spark-apps/spark-1.6.1/core/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/launcher/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/network/common/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/network/shuffle/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/unsafe/target/scala-2.10/classes:/home/ashish/.ivy2/cache/org.spark-project.spark/unused/jars/unused-1.0.0.jar:/home/ashish/.ivy2/cache/com.google.guava/guava/bundles/guava-14.0.1.jar:/home/ashish/.ivy2/cache/io.netty/netty-all/jars/netty-all-4.0.29.Final.jar:/home/ashish/.ivy2/cache/org.fusesource.leveldbjni/leveldbjni-all/bundles/leveldbjni-all-1.8.jar:/home/ashish/.ivy2/cache/com.fasterxml.jackson.core/jackson-databind/bundles/jackson-databind-2.4.4.jar:/home/ashish/.ivy2/cache/com.fasterxml.jackson.core/jackson-annotations/bundles/jackson-annotations-2.4.4.jar:/home/ashish/.ivy2/cache/com.fasterxml.jackson.core/jackson-core/bundles/jackson-......and
many other jars...
[error] [error] last tree to typer:
Literal(Constant(collection.mutable.Map)) [error]
symbol: null [error] symbol definition: null [error]
tpe: Class(classOf[scala.collection.mutable.Map]) [error]
symbol owners: [error] context owners: package package ->
package random [error] [error] == Enclosing template or
block == [error] [error] Template( // val :
in package random,
tree.tpe=org.apache.spark.util.random.package.type [error]
"java.lang.Object" // parents [error] ValDef( [error]
private [error] "_" [error] [error]
[error] ) [error] DefDef( // def ():
org.apache.spark.util.random.package.type in package random
[error] [error] "" [error]
[] [error] List(Nil) [error] //
tree.tpe=org.apache.spark.util.random.package.type [error]
Block( // tree.tpe=Unit [error] Apply( // def ():
Object in class Object, tree.tpe=Object [error]
package.super."" // def (): Object in class Object,
tree.tpe=()Object [error] Nil [error] )
[error] () [error] ) [error] ) [error]
) [error] [error] == Expanded type of tree == [error]
[error] ConstantType(value = Constant(collection.mutable.Map))
[error] [error] uncaught exception during compilation:
java.io.IOException [error] File name too long [warn] 45
warnings found [error] two errors found [error]
(core/compile:compile) Compilation failed [error] Total time:
5598 s, completed 5 Apr, 2016 9:06:50 AM
Where I'm getting wrong?
You should build Spark with Maven...
download the source and run ./bin/mvn clean package
Probably similar to http://apache-spark-user-list.1001560.n3.nabble.com/spark-github-source-build-error-td10532.html
Try sudo sbt/sbt clean assembly

sbt assembly can't find type in the same package

I have the code shown here
http://scastie.org/10156
report the error
when running sbt 0.13.8
plugin assembly
I get
[error] /home/xx/workspace/sync-hsae-middleware/src/test/scala/com/example/MyServiceSpec.scala:8: not found: type MyService
[error] class MyServiceSpec extends Specification with Specs2RouteTest with MyService {
[error] ^
[error] /home/xx/workspace/sync-hsae-middleware/src/test/scala/com/example/MyServiceSpec.scala:14: not found: value myRoute
[error] Get() ~> myRoute ~> check {
[error] ^
[error] two errors found
[error] (test:compileIncremental) Compilation failed
[error] Total time: 5 s, completed 10/06/2015 15:00:15
*/
I can't figure out what is the issue because when I use
sbt run it works well.

Unable to verify subset of arguments passed to mocked method in specs2

Here's a simple specification reproducing the issue:
package ro.igstan.learning
import org.specs2.matcher.ThrownExpectations
import org.specs2.mock.Mockito
import org.specs2.mutable.Specification
import org.specs2.specification.Scope
class LearningSpecs2 extends Specification {
trait Logger {
def error(a: String, b: String)
}
trait mocks extends Mockito with Scope with ThrownExpectations {
val mockedLogger = mock[Logger]
}
"mocking" should {
// passes
"be able to verify arguments passed to mocks" in new mocks {
mockedLogger.error("message", "foo")
got {
one(mockedLogger).error("message", "foo")
}
}
// fails
"be able to *partially* verify arguments passed to mocks" in new mocks {
mockedLogger.error("message", "foo")
got {
one(mockedLogger).error(any, ===("foo"))
}
}
}
}
This is the output:
[info] mocking should
[info] + be able to verify arguments passed to mocks
[error] ! Fragment evaluation error
[error] ThrowableException: org/hamcrest/TypeSafeMatcher (FutureTask.java:303)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.argThat(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply$mcV$sp(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.got(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.<init>(LearningSpecs2.scala:30)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.argThat(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply$mcV$sp(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.got(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.<init>(LearningSpecs2.scala:30)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
[error] org/hamcrest/TypeSafeMatcher
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.argThat(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply$mcV$sp(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.got(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.<init>(LearningSpecs2.scala:30)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
[error] org.hamcrest.TypeSafeMatcher
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.argThat(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply$mcV$sp(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2$$anonfun$2.apply(LearningSpecs2.scala:31)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.got(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3$$anon$2.<init>(LearningSpecs2.scala:30)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
[error] ro.igstan.learning.LearningSpecs2$$anonfun$3$$anonfun$apply$3.apply(LearningSpecs2.scala:27)
And here are the library versions as reported by the SBT dependency graph plugin:
[info] default:learning_2.9.1:1.0.0
[info] +-junit:junit:4.7
[info] +-org.mockito:mockito-all:1.9.0
[info] +-org.scala-lang:scala-library:2.9.1
[info] +-org.specs2:specs2_2.9.1:1.11
[info] +-org.specs2:specs2-scalaz-core_2.9.1:6.0.1
[info] +-org.scala-lang:scala-library:2.9.1
Any idea what's going on?
Fixed. I had to explicitly specify the Hamcrest version:
libraryDependencies ++= Seq(
"org.specs2" %% "specs2" % "1.11" ,
"org.mockito" % "mockito-all" % "1.9.0",
"org.hamcrest" % "hamcrest-all" % "1.3", // <- this was missing
"junit" % "junit" % "4.7"
)