I can't seem to find why scala packages are being "thought of" by maven as a member of package org.apache.kafka.streams.scala. I usually use sbt but I have to use maven on this one. Any help would be appreciated.
/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/bin/java -Dmaven.multiModuleProjectDirectory=/Users/jcena1/IdeaProjects/datapipe-scala-merge-transform "-Dmaven.home=/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3" "-Dclassworlds.conf=/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3/bin/m2.conf" "-javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=65172:/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3/boot/plexus-classworlds-2.5.2.jar" org.codehaus.classworlds.Launcher -Didea.version=2018.2.1 install
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for io.confluent:Jointransform:jar:4.1.1
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: org.scala-lang:scala-library:jar -> duplicate declaration of version ${scala.version} # io.confluent:Jointransform:[unknown-version], /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/pom.xml, line 259, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.13
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 13
[INFO] os.detected.classifier: osx-x86_64
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Jointransform 4.1.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) # Jointransform ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) # Jointransform ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.10:add-source (add-source) # Jointransform ---
[INFO] Source directory: /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala added.
[INFO] Source directory: /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/generated-sources added.
[INFO]
[INFO] --- avro-maven-plugin:1.8.2:schema (default) # Jointransform ---
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) # Jointransform ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 28 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) # Jointransform ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 11 source files to /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/classes
[INFO]
[INFO] --- scala-maven-plugin:3.2.1:compile (default) # Jointransform ---
[INFO] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala:-1: info: compiling
[INFO] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/generated-sources:-1: info: compiling
[INFO] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/generated-sources/annotations:-1: info: compiling
[INFO] Compiling 16 source files to /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/target/classes at 1536749088682
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Filtertransform.scala:10: error: object language is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.language.implicitConversions
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Filtertransform.scala:14: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Filtertransform.scala:75: error: Symbol 'type org.apache.kafka.streams.Consumed' is missing from the classpath.
[ERROR] This symbol is required by 'value com.lightbend.kafka.scala.streams.StreamsBuilderS.consumed'.
[ERROR] Make sure that type Consumed is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[ERROR] A full rebuild may help if 'StreamsBuilderS.class' was compiled against an incompatible version of org.apache.kafka.streams.
[ERROR] val stream: KStreamS[String, Array[Byte]] = builder.stream(inputTopic)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:12: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.JavaConverters._
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:11: error: object language is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.language.implicitConversions
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:20: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:21: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable.ListBuffer
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:39: error: Symbol 'type org.apache.kafka.streams.Consumed' is missing from the classpath.
[ERROR] This symbol is required by 'method com.lightbend.kafka.scala.streams.ImplicitConversions.consumedFromSerde'.
[ERROR] Make sure that type Consumed is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[ERROR] A full rebuild may help if 'ImplicitConversions.class' was compiled against an incompatible version of org.apache.kafka.streams.
[ERROR] (out:+ s)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:57: error: value asScala is not a member of java.util.List[String]
[ERROR] possible cause: maybe a semicolon is missing before `value asScala'?
[ERROR] .asScala
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:147: error: not found: value Consumed
[ERROR] implicit val c = Consumed.`with`(Serdes.String(), Serdes.ByteArray())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:149: error: polymorphic expression cannot be instantiated to expected type;
[ERROR] found : [K, V]org.apache.kafka.streams.kstream.KTable[K,V]
[ERROR] required: org.apache.kafka.streams.scala.kstream.KTable[String,Array[Byte]]
[ERROR] val table: KTable[String, Array[Byte]] = builder.table(inputTopic)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:156: error: not found: value Serialized
[ERROR] implicit val sb = Serialized.`with`(Serdes.String(), Serdes.ByteArray())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:157: error: not found: value Serialized
[ERROR] implicit val ss = Serialized.`with`(Serdes.String(), Serdes.String())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:158: error: not found: value Produced
[ERROR] implicit val p = Produced.`with`(Serdes.String(),Serdes.String())
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:241: error: value asScala is not a member of java.util.List[String]
[ERROR] possible cause: maybe a semicolon is missing before `value asScala'?
[ERROR] .asScala
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:245: error: value asScala is not a member of java.util.List[String]
[ERROR] val newValue = getValFromJSONMessage(lv, aggregateColumnList.asScala.toList.head)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:232: error: diverging implicit expansion for type org.apache.kafka.streams.kstream.Serialized[KR,VR]
[ERROR] starting with method consumedFromSerde in object ImplicitConversions
[ERROR] val groupedK = table.groupBy{ (key: String, value: Array[Byte]) =>
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:252: error: not found: type Aggregator
[ERROR] val newAggregator: Aggregator[String, Array[Byte], Array[Byte]] = new Aggregator[String, Array[Byte], Array[Byte]] {
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/GroupBytransform.scala:252: error: not found: type Aggregator
[ERROR] val newAggregator: Aggregator[String, Array[Byte], Array[Byte]] = new Aggregator[String, Array[Byte], Array[Byte]] {
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:10: error: object language is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.language.implicitConversions
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:11: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.JavaConverters._
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:18: error: object collection is not a member of package org.apache.kafka.streams.scala
[ERROR] import scala.collection.mutable.ListBuffer
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:37: error: not found: type ListBuffer
[ERROR] val lst = new ListBuffer[String]
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:84: error: value asScala is not a member of java.util.List[String]
[ERROR] Note: implicit value j is not applicable here because it comes after the application point and it lacks an explicit result type
[ERROR] val leftRekeyColumnList = if(rekeyLeftNeeded) joinStep.getStringList("joinOn.leftFields").asScala.toList else List.empty
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:85: error: value asScala is not a member of java.util.List[String]
[ERROR] Note: implicit value j is not applicable here because it comes after the application point and it lacks an explicit result type
[ERROR] val rightRekeyColumnList = if(rekeyRightNeeded) joinStep.getStringList("joinOn.rightFields").asScala.toList else List.empty
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:96: error: value asScala is not a member of java.util.List[String]
[ERROR] Note: implicit value j is not applicable here because it comes after the application point and it lacks an explicit result type
[ERROR] if(!JoinUtils.validateFilterExpression(filterExpression.asScala.toList)) {
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:147: error: value asScala is not a member of java.util.List[String]
[ERROR] JoinUtils.joinValues(lv, rv, node, outputFields.asScala.toList)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:157: error: value asScala is not a member of java.util.List[String]
[ERROR] val filterPredicate = JoinUtils.buildPostJoinPredicate(filterExpression.asScala.toList, lv)
[ERROR] ^
[ERROR] /Users/jcena1/IdeaProjects/datapipe-scala-merge-transform/src/main/scala/acme/Jointransform.scala:165: error: value asScala is not a member of java.util.List[String]
[ERROR] val ba: Array[Byte] = JoinUtils.removeEntryFromJsonNode(extraFields.asScala.toList, value)
[ERROR] ^
[ERROR] 29 errors found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.937 s
[INFO] Finished at: 2018-09-12T06:44:53-04:00
[INFO] Final Memory: 35M/376M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.1:compile (default) on project Jointransform: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Process finished with exit code 1
I needed to prefix _root_ to all the Scala core packages like for example
import _root_.scala.collection.JavaConverters._
Related
I am trying to import a module from another project and did as following:
As you can see on the image, the imported library is /home/developer/...kafka-api.
I am using the importing library in testing.
When I compile my spec files with statement test:compile and I've got following error:
[IJ]sbt:auth_stream> test:compile
[info] Compiling 3 Scala sources to /home/developer/Desktop/microservices/bary/auth-stream/target/scala-2.12/test-classes ...
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:14:20: object kkapi is not a member of package io.khinkali
[error] import io.khinkali.kkapi.consumer.{KkConsumer, KkConsumerConfig, KkConsumerCreator}
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:15:20: object kkapi is not a member of package io.khinkali
[error] import io.khinkali.kkapi.producer.{KkProducer, KkProducerCreator, MaxBlockMsConfig}
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:24:56: not found: value KkConsumer
[error] private val consumer: IO[Consumer[String, String]] = KkConsumer.create(createConsumer())
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:52:5: not found: type KkConsumerCreator
[error] : KkConsumerCreator
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:25:56: not found: value KkProducer
[error] private val producer: IO[Producer[String, String]] = KkProducer.create(createProducer())
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:46:5: not found: type KkProducerCreator
[error] : KkProducerCreator
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:47:5: not found: value KkProducerCreator
[error] = KkProducerCreator(sys.env.get("KAFKA_SERVER").get,
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:49:10: not found: value MaxBlockMsConfig
[error] List(MaxBlockMsConfig(2000)))
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:53:5: not found: value KkConsumerCreator
[error] = KkConsumerCreator(sys.env.get("KAFKA_SERVER").get,
[error] ^
[error] /home/developer/Desktop/microservices/bary/auth-stream/src/test/scala/io/khinkali/auth/AppSpec.scala:57:16: not found: type KkConsumerConfig
[error] List.empty[KkConsumerConfig])
[error] ^
[error] 10 errors found
[error] (test:compileIncremental) Compilation failed
What am I doing wrong?
Hint, that the package of both project starts with the name, namely:
The current project:
The imported project:
As you can see, the name differs only at the end. Could it be the problem?
What I am trying to approach is, to use a function for kafka-api project.
I'm trying to use chisel 3.
I tried to test GCD.scala file in the chisel project template repo using sbt test and sbt "test-only example.GCD" commands following the answer to a previous question. But this gives an error(s) that I cannot find the reason for. I didn't do any changes to the build.sbt file or repo layout. I'm posting only the last part of the error message since it is very long and repetitive.
[info] Loading project definition from /home/isuru/fyp/ChiselProjects/TrialProject/project
[info] Set current project to chisel-module-template (in build file:/home/isuru/fyp/ChiselProjects/TrialProject/)
[info] Compiling 1 Scala source to /home/isuru/fyp/ChiselProjects/TrialProject/target/scala-2.11/classes...
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:5: not found: object Chisel3
[error] import Chisel3._
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:7: not found: type Module
[error] class GCD extends Module {
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:8: not found: type Bundle
[error] val io = new Bundle {
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:9: not found: value UInt
[error] val a = UInt(INPUT, 16)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:9: not found: value INPUT
[error] val a = UInt(INPUT, 16)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:10: not found: value UInt
[error] val b = UInt(INPUT, 16)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:10: not found: value INPUT
[error] val b = UInt(INPUT, 16)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:11: not found: value Bool
[error] val e = Bool(INPUT)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:11: not found: value INPUT
[error] val e = Bool(INPUT)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:12: not found: value UInt
[error] val z = UInt(OUTPUT, 16)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:12: not found: value OUTPUT
[error] val z = UInt(OUTPUT, 16)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:13: not found: value Bool
[error] val v = Bool(OUTPUT)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:13: not found: value OUTPUT
[error] val v = Bool(OUTPUT)
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:15: not found: value Reg
[error] val x = Reg(UInt())
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:15: not found: value UInt
[error] val x = Reg(UInt())
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:16: not found: value Reg
[error] val y = Reg(UInt())
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:16: not found: value UInt
[error] val y = Reg(UInt())
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:17: not found: value when
[error] when (x > y) { x := x - y }
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:18: not found: value unless
[error] unless (x > y) { y := y - x }
[error] ^
[error] /home/isuru/fyp/ChiselProjects/TrialProject/src/main/scala/example/GCD.scala:19: not found: value when
[error] when (io.e) { x := io.a; y := io.b }
[error] ^
[error] 20 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 2 s, completed Dec 1, 2016 8:26:25 PM
The errors you have shown suggest that sbt is somehow not finding Chisel, could you by chance show the full list of errors (especially early on ones)? With the following sequence of commands I am unable to reproduce the errors you are seeing:
git clone git#github.com:ucb-bar/chisel-template.git
cd chisel-template
sbt test
It is not the cause of this issue, but to run the test in chisel-template you should actually run sbt "test-only examples.test.GCDTester". example.GCD is the top of the design, but to run the test you have to refer to the Tester class in src/test/scala/examples/test/GCDUnitTest.scala.
I just encountered this same problem when making my own chisel project. However it was not that the import chisel3._ was wrong. The problem I had was I did not have a build.sbt file included in my directory.
I found my solution here.
https://chisel.eecs.berkeley.edu/2.0.6/getting-started.html
I'm building Apache spark source code in ubuntu 14.04.4 (spark version: 1.6.0 with Scala code runner version 2.10.4) with command
sudo sbt/sbt assembly
and getting the following error,
[warn] def deleteRecursively(dir: TachyonFile, client: TachyonFS) {
[warn] ^
[error] [error] while compiling:
/home/ashish/spark-apps/spark-1.6.1/core/src/main/scala/org/apache/spark/util/random/package.scala
[error] during phase: jvm [error] library
version: version 2.10.5 [error] compiler version: version
2.10.5 [error] reconstructed args: -deprecation -Xplugin:/home/ashish/.ivy2/cache/org.spark-project/genjavadoc-plugin_2.10.5/jars/genjavadoc-plugin_2.10.5-0.9-spark0.jar
-feature -P:genjavadoc:out=/home/ashish/spark-apps/spark-1.6.1/core/target/java -classpath /home/ashish/spark-apps/spark-1.6.1/core/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/launcher/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/network/common/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/network/shuffle/target/scala-2.10/classes:/home/ashish/spark-apps/spark-1.6.1/unsafe/target/scala-2.10/classes:/home/ashish/.ivy2/cache/org.spark-project.spark/unused/jars/unused-1.0.0.jar:/home/ashish/.ivy2/cache/com.google.guava/guava/bundles/guava-14.0.1.jar:/home/ashish/.ivy2/cache/io.netty/netty-all/jars/netty-all-4.0.29.Final.jar:/home/ashish/.ivy2/cache/org.fusesource.leveldbjni/leveldbjni-all/bundles/leveldbjni-all-1.8.jar:/home/ashish/.ivy2/cache/com.fasterxml.jackson.core/jackson-databind/bundles/jackson-databind-2.4.4.jar:/home/ashish/.ivy2/cache/com.fasterxml.jackson.core/jackson-annotations/bundles/jackson-annotations-2.4.4.jar:/home/ashish/.ivy2/cache/com.fasterxml.jackson.core/jackson-core/bundles/jackson-......and
many other jars...
[error] [error] last tree to typer:
Literal(Constant(collection.mutable.Map)) [error]
symbol: null [error] symbol definition: null [error]
tpe: Class(classOf[scala.collection.mutable.Map]) [error]
symbol owners: [error] context owners: package package ->
package random [error] [error] == Enclosing template or
block == [error] [error] Template( // val :
in package random,
tree.tpe=org.apache.spark.util.random.package.type [error]
"java.lang.Object" // parents [error] ValDef( [error]
private [error] "_" [error] [error]
[error] ) [error] DefDef( // def ():
org.apache.spark.util.random.package.type in package random
[error] [error] "" [error]
[] [error] List(Nil) [error] //
tree.tpe=org.apache.spark.util.random.package.type [error]
Block( // tree.tpe=Unit [error] Apply( // def ():
Object in class Object, tree.tpe=Object [error]
package.super."" // def (): Object in class Object,
tree.tpe=()Object [error] Nil [error] )
[error] () [error] ) [error] ) [error]
) [error] [error] == Expanded type of tree == [error]
[error] ConstantType(value = Constant(collection.mutable.Map))
[error] [error] uncaught exception during compilation:
java.io.IOException [error] File name too long [warn] 45
warnings found [error] two errors found [error]
(core/compile:compile) Compilation failed [error] Total time:
5598 s, completed 5 Apr, 2016 9:06:50 AM
Where I'm getting wrong?
You should build Spark with Maven...
download the source and run ./bin/mvn clean package
Probably similar to http://apache-spark-user-list.1001560.n3.nabble.com/spark-github-source-build-error-td10532.html
Try sudo sbt/sbt clean assembly
There is all of the code in my project:
package fileSearcher
import org.scalatest.FlatSpec
class FilterCheckerTests extends org.scalatest.FlatSpec {
"Foo" should
"not do terrible things" in {
assert(1 == 1)
}
}
sbt test crashes with scala.MatchError (full details below).
What am I doing wrong?
[info] Compiling 1 Scala source to C:\scala\course\FileSearcher\target\scala-2.1
0\test-classes...
[error]
[error] while compiling: C:\scala\course\FileSearcher\src\test\scala\fileSe
archer\FilterCheckerTests.scala
[error] during phase: typer
[error] library version: version 2.10.4
[error] compiler version: version 2.10.4
[error] reconstructed args: -classpath C:\scala\course\FileSearcher\target\sca
la-2.10\test-classes;C:\scala\course\FileSearcher\target\scala-2.10\classes;C:\U
sers\Max\.ivy2\cache\org.scalatest\scalatest_2.11\bundles\scalatest_2.11-2.2.4.j
ar;C:\Users\Max\.ivy2\cache\org.scala-lang\scala-reflect\jars\scala-reflect-2.11
.2.jar;C:\Users\Max\.ivy2\cache\org.scala-lang.modules\scala-xml_2.11\bundles\sc
ala-xml_2.11-1.0.2.jar;C:\Users\Max\.ivy2\cache\com.novocode\junit-interface\jar
s\junit-interface-0.11.jar;C:\Users\Max\.ivy2\cache\junit\junit\jars\junit-4.11.
jar;C:\Users\Max\.ivy2\cache\org.hamcrest\hamcrest-core\jars\hamcrest-core-1.3.j
ar;C:\Users\Max\.ivy2\cache\org.scala-sbt\test-interface\jars\test-interface-1.0
.jar -bootclasspath C:\Program Files\Java\jdk1.8.0_20\jre\lib\resources.jar;C:\P
rogram Files\Java\jdk1.8.0_20\jre\lib\rt.jar;C:\Program Files\Java\jdk1.8.0_20\j
re\lib\sunrsasign.jar;C:\Program Files\Java\jdk1.8.0_20\jre\lib\jsse.jar;C:\Prog
ram Files\Java\jdk1.8.0_20\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_20\jre
\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_20\jre\lib\jfr.jar;C:\Program F
iles\Java\jdk1.8.0_20\jre\classes;C:\Users\Max\.ivy2\cache\org.scala-lang\scala-
library\jars\scala-library-2.10.4.jar
[error]
[error] last tree to typer: Literal(Constant(true))
[error] symbol: null
[error] symbol definition: null
[error] tpe: Boolean(true)
[error] symbol owners:
[error] context owners: value <local FilterCheckerTests> -> class FilterCh
eckerTests -> package fileSearcher
[error]
[error] == Enclosing template or block ==
[error]
[error] Template( // val <local FilterCheckerTests>: <notype> in class FilterChe
ckerTests
[error] "org.scalatest.FlatSpec" // parents
[error] ValDef(
[error] private
[error] "_"
[error] <tpt>
[error] <empty>
[error] )
[error] // 2 statements
[error] DefDef( // def <init>(): fileSearcher.FilterCheckerTests in class Filt
erCheckerTests
[error] <method>
[error] "<init>"
[error] []
[error] List(Nil)
[error] <tpt> // tree.tpe=fileSearcher.FilterCheckerTests
[error] Block( // tree.tpe=Unit
[error] Apply( // def <init>(): org.scalatest.FlatSpec in class FlatSpec,
tree.tpe=org.scalatest.FlatSpec
[error] FilterCheckerTests.super."<init>" // def <init>(): org.scalatest
.FlatSpec in class FlatSpec, tree.tpe=()org.scalatest.FlatSpec
[error] Nil
[error] )
[error] ()
[error] )
[error] )
[error] Apply(
[error] "Foo".should("not do terrible things")."in"
[error] Apply(
[error] "assert"
[error] Apply( // def ==(x: Int): Boolean in class Int, tree.tpe=Boolean(t
rue)
[error] 1."$eq$eq" // def ==(x: Int): Boolean in class Int, tree.tpe=(x:
Int)Boolean
[error] 1
[error] )
[error] )
[error] )
[error] )
[error]
[error] == Expanded type of tree ==
[error]
[error] ConstantType(value = Constant(true))
[error]
[error] uncaught exception during compilation: scala.MatchError
[trace] Stack trace suppressed: run last test:compile for the full output.
[error] (test:compile) scala.MatchError: false (of class scala.reflect.internal.
Trees$Literal)
[error] Total time: 0 s, completed Jun 20, 2015 11:07:15 AM
1. Waiting for source changes... (press enter to interrupt)
As you can see by looking at the classpath, which is printed by the compiler, you mixed Scala 2.10 with libraries for 2.11. Given that major versions of Scala are binary incompatible, this can never work.
This can be fixed with scalaVersion := "2.11.5" or by setting all dependencies to use the 2.10 versions, which can be done with libraryDependencies += "group" %% "libName" % "version", where %% means that sbt automatically uses the correct library version.
For anyone getting a similar error (as I did) running Scala 2.10 using Maven instead of sbt, the solution would simply be to change the Maven dependency suggested in the ScalaTest website from
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.11</artifactId>
<version>3.0.0</version>
<scope>test</scope>
</dependency>
to
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.10</artifactId>
<version>3.0.0</version>
<scope>test</scope>
</dependency>
I have the code shown here
http://scastie.org/10156
report the error
when running sbt 0.13.8
plugin assembly
I get
[error] /home/xx/workspace/sync-hsae-middleware/src/test/scala/com/example/MyServiceSpec.scala:8: not found: type MyService
[error] class MyServiceSpec extends Specification with Specs2RouteTest with MyService {
[error] ^
[error] /home/xx/workspace/sync-hsae-middleware/src/test/scala/com/example/MyServiceSpec.scala:14: not found: value myRoute
[error] Get() ~> myRoute ~> check {
[error] ^
[error] two errors found
[error] (test:compileIncremental) Compilation failed
[error] Total time: 5 s, completed 10/06/2015 15:00:15
*/
I can't figure out what is the issue because when I use
sbt run it works well.