parboiled2 and Spray cause conflicting cross-version suffixes - scala

I'm trying to add parboiled2 as a dependency to my project, and follow the Calculator example but it conflicts with spray.
My current build.sbt file includes:
"io.spray" %% "spray-json" % "1.3.1" withSources() withJavadoc(),
"io.spray" %% "spray-can" % sprayV withSources() withJavadoc(),
"io.spray" %% "spray-routing" % sprayV withSources() withJavadoc(),
"io.spray" %% "spray-testkit" % sprayV % "test" withSources() withJavadoc(),
When I add
"org.parboiled" %% "parboiled" % "2.0.1" withSources() withJavadoc(),
I get
[error] Modules were resolved with conflicting cross-version suffixes in {file:/blar/blar}blar-blar:
[error] com.chuusai:shapeless _2.10.4, _2.10
[error] org.scalamacros:quasiquotes _2.10, _2.10.3
java.lang.RuntimeException: Conflicting cross-version suffixes in: com.chuusai:shapeless, org.scalamacros:quasiquotes
So did some googling, and as usual people suggest using the exclude directive of SBT (which I don't believe makes logical sense as it will inevitably result in problems like ClassNotFoundException and NoSuchMethodError). I tried it nevertheless:
"org.parboiled" %% "parboiled" % "2.0.1" withSources() withJavadoc()
exclude("com.chuusai", "shapeless_2.10.4") exclude("org.scalamacros", "quasiquotes_2.10")
And surprise surprise when I try to run assembly I get
[error] java.lang.ClassNotFoundException: scala.quasiquotes.QuasiquoteCompat$
I also get a load more errors:
[error] bad symbolic reference
. A signature in RuleDSLBasics.class refers to term internal
[error] in package scala.reflect which is not available.
[error] It may be completely missing from the current classpath, or the version on
[error] the classpath might be incompatible with the version used when compiling RuleDSLBasics.class.
[error] error while loading RuleDSLBasics, Missing dependency 'bad symbolic reference. A signature in RuleDSLBasics.class refers to term annotations
[error] in value scala.reflect.internal which is not available.
[error] It may be completely missing from the current classpath, or the version on
[error] the classpath might be incompatible with the version used when compiling RuleDSLBasics.class.', required by ...
If I comment out the Spray dependencies (and code) I can successfully assembly a jar with the Calculator example.
This is a major show stopper for me using parboiled2. Is there some way other than exclude directives in order to make this stuff work? Does anyone have a working build file with both Spray and parboiled2? If someone has managed to get around dependency hell with onejar or ProGuard I'd love it if they could explain just how.
UPDATE:
My build file:
resolvers ++= Seq(
"Concurrent Maven Repo" at "http://conjars.org/repo",
"spray repo" at "http://repo.spray.io"
)
val akkaV = "2.3.6"
val sprayV = "1.3.2"
libraryDependencies ++= Seq(
"org.parboiled" %% "parboiled" % "2.0.1" withSources() withJavadoc(),
// Causes org.scalamacros:quasiquotes _2.10, _2.10.3 cross-version problem
"io.spray" %% "spray-testkit" % sprayV % "test" withSources() withJavadoc(),
// Causes com.chuusai:shapeless _2.10.4, _2.10 cross-version problem
"io.spray" %% "spray-routing" % sprayV withSources() withJavadoc()
)
scalaVersion := "2.10.4"
javaOptions ++= Seq("-target", "1.8", "-source", "1.8")
organization := domain + "." + companyName
Plugins file:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.6.0")
Scala 2.11
It seems "simply" moving 2.11 may solve this, but this isn't always that easy if one has some other libraries that are not 2.11 ready. Nevertheless I tried it, and the second I added parboiled2 as a dependency it broke my build again, sigh, here is the new problem: Parboiled2 causes "missing or invalid dependency detected while loading class file 'Prepender.class'"

The same answer as for Scala 2.11 should work here as well: replace spray-routing with spray-routing-shapeless2.

I would have to look at the whole build.sbt file to understand what is happening but I can use the dependencies you mention without problem with sbt 0.13.7 and scala 2.11.
What sbt version? What Scala version? 2.10 it seems. Why? Are you stuck with those versions?
For reference, here are the build.sbt, plugins.sbt and build.properties files I used to test this scenario.
https://gist.github.com/fedesilva/ddd28a5b592eb88f41ae
Hope that helps

Related

upgrade from Scala 2.11.8 to 2.12.10 build fails at sbt due to conflicting cross-version suffixes

I am trying to upgrade my Scala version from 2.11.8 to 2.12.10. I made following changes in my sbt file.
"org.apache.spark" %% "spark-core" % "2.4.7" % "provided",
"org.apache.spark" %% "spark-sql" % "2.4.7" % "provided",
"com.holdenkarau" %% "spark-testing-base" % "3.1.2_1.1.0" % "test"
when I am build the sbt file, I am getting following error
[error] Modules were resolved with conflicting cross-version suffixes in ProjectRef(uri("file:/Users/user/IdeaProjects/project/"), "root"):
[error] io.reactivex:rxscala _2.12, _2.11
Tried following possible ways. but no luck.
("io.reactivex" % "rxscala_2.12" % "0.27.0").force().exclude("io.reactivex","rxscala_2.11")
2.Removed Scala verion=2.11.8 from File->project structure->global libraries.
Any Help will be very useful.
It would be best if you could post your entire build.sbt file so it could be reproduced, but as a starting point, the dependency spark-testing-base is pointing to the wrong Spark version. From the documentation:
So you include com.holdenkarau.spark-testing-base [spark_version]_1.0.0 and extend
Based on the information you have provided you should be using:
"com.holdenkarau" %% "spark-testing-base" % "2.4.7_1.1.0" % "test"

Scala SBT is not able to download dependencies

I am new to scala and Akka. I have created a new project and below is my build.sbt file.
name := "akka_essentials"
version := "0.1"
scalaVersion := "2.13.4"
val akkaVersion = "2.5.13"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-testkit" % akkaVersion,
"com.scalatest" %% "scalatest" % "3.0.5"
)
The sbt tool is not able to download dependencies. The error log is as below:
[error] stack trace is suppressed; run 'last update' for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading com.typesafe.akka:akka-testkit_2.13:2.5.13
[error] Not found
[error] Not found
[error] not found: C:\Users\Anand\.ivy2\local\com.typesafe.akka\akka-testkit_2.13\2.5.13\ivys\ivy.xml
[error] not found: https://repo1.maven.org/maven2/com/typesafe/akka/akka-testkit_2.13/2.5.13/akka-testkit_2.13-2.5.13.pom
[error] Error downloading com.scalatest:scalatest_2.13:3.0.5
[error] Not found
[error] Not found
[error] not found: C:\Users\Anand\.ivy2\local\com.scalatest\scalatest_2.13\3.0.5\ivys\ivy.xml
[error] not found: https://repo1.maven.org/maven2/com/scalatest/scalatest_2.13/3.0.5/scalatest_2.13-3.0.5.pom
[error] Error downloading com.typesafe.akka:akka-actor_2.13:2.5.13
[error] Not found
[error] Not found
[error] not found: C:\Users\Anand\.ivy2\local\com.typesafe.akka\akka-actor_2.13\2.5.13\ivys\ivy.xml
[error] not found: https://repo1.maven.org/maven2/com/typesafe/akka/akka-actor_2.13/2.5.13/akka-actor_2.13-2.5.13.pom
I am very new to scala. I have not defined anything like ivy.xml. I am from java background and generally use gradle.
Please advise to resolve this.
Akka 2.5.13 has not been cross published for Scala 2.13 as you can check on Maven.
The earliest version supporting 2.13 (release, not milestone or RC) is 2.5.23. You can see the version matrix for Actors here.
So you need to use newer Akka or older Scala.
With Scalatest you used wrong organization. It's "org.scalatest" not "com.scalatest".
If you have doubts about dependency resoultion check Maven first.
I did the following changes to make it work:
name := "project"
version := "0.1"
scalaVersion := "2.12.7"
val akkaVersion = "2.5.13"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-testkit" % akkaVersion
// "com.scalatest" %% "scalatest" % "3.0.5"
)

Why does sbt update fail with "Conflicting cross-version suffixes" with Spark GraphX?

Here is my sbt for spark with scala on Intellij
version := "0.1"
scalaVersion := "2.11.11"
// https://mvnrepository.com/artifact/org.apache.spark/spark-graphx_2.10
libraryDependencies += "org.apache.spark" % "spark-graphx_2.10" % "2.1.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11
libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
I get the following error
[error] (*:update) Conflicting cross-version suffixes in:org.apache.spark:spark-launcher, org.json4s:json4s-ast, org.apache.spark:spark-network-shuffle, org.scalatest:scalatest, com.twitter:chill, org.json4s:json4s-jackson, com.fasterxml.jackson.module:jackson-module-scala, org.json4s:json4s-core,org.apache.spark:spark-unsafe, org.apache.spark:spark-tags, org.apache.spark:spark-core, org.apache.spark:spark-network-common
[error] (*:ssExtractDependencies) Conflicting cross-version suffixes in: org.apache.spark:spark-launcher, org.json4s:json4s-ast, org.apache.spark:spark-network-shuffle, org.scalatest:scalatest, com.twitter:chill, org.json4s:json4s-jackson, com.fasterxml.jackson.module:jackson-module-scala, org.json4s:json4s-core, org.apache.spark:spark-unsafe, org.apache.spark:spark-tags, org.apache.spark:spark-core, org.apache.spark:spark-network-common
Can anyone let me know how to correct this. To me it looks like spark-graphx is contradicting with spark_core. How do I find the correct SBT file? or what should i do to get this sbt file working. I am ok with lowering the version of graphx as well as spark but would prefer to lower the version of spark.
In you sbt file, the dependencies are called spark-_. You're using different scala version in your graphX dependency.
If you use Spark 2.x I recommend use scala 2.11 so you only have to change graphx dependency.
libraryDependencies += "org.apache.spark" % "spark-graphx_2.11" % "2.1.0"
Anyway, you don't need to write scala version in every dependency. Sbt will infer according scalaVersion value if you write two percentage sign %%. Here, you can see an example.

Scala sbt: Multiple dependencies in sbt

I am a new user to Scala, following the way to create a scala sbt project.
https://www.youtube.com/watch?v=Ok7gYD1VbNw
After adding
libraryDependencies += "org.scalatest" % "scalatest_2.11" % "2.2.4" % "test"
to build.sbt, and refreshed the project, I got this msg.
[warn] Multiple dependencies with the same organization/name but different versions. To avoid conflict, pick one version:
[warn] * org.scala-lang:scala-reflect:(2.11.2, 2.11.7)
[warn] * org.scala-lang.modules:scala-xml_2.11:(1.0.2, 1.0.4)
And in build.sbt, thw word 'scalatest' is red that means it's an unsolved dependencies.
Should I change something in Project Setting, like scala sdk?
Best Regard!
You could regard adding those dependencies:
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % "2.11.7",
"org.scala-lang.modules" % "scala-xml_2.11" % "1.0.4"
)
It forces compiler to choose concrete version of libraries. It solves problem for me.
I was able to resolve this by excluding these from the scalatest dependency.
libraryDependencies ++= Seq(
"org.scalatest" % "scalatest_2.11" % "2.2.4" % "test"
exclude("org.scala-lang", "scala-reflect")
exclude("org.scala-lang.modules", "scala-xml_2.11")
)

scala Either and akka.dispatch Future missing while compiling

I am trying to create a REST service using Spray Servlet, but does not compile.
Below you can find my build.sbt:
name := "someservice"
version := "0.0.1"
scalaVersion := "2.10.0"
resolvers += "spray repo" at "http://repo.spray.io"
libraryDependencies += "io.spray" % "spray-servlet" % "1.0-M3"
libraryDependencies ++= Seq(
"io.spray" % "spray-servlet" % "1.0-M3",
"io.spray" % "spray-util" % "1.0-M3",
"io.spray" % "spray-http" % "1.1-M7",
"com.typesafe.akka" %% "akka-actor" % "2.1.0",
"org.specs2" %% "specs2" % "1.13" % "test",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" artifacts Artifact("javax.servlet", "jar", "jar")
)
There error I get com 'sbt update compile' is:
[error] bad symbolic reference. A signature in package.class refers to term Either
[error] in package scala which is not available.
[error] It may be completely missing from the current classpath, or the version on
[error] the classpath might be incompatible with the version used when compiling
[error] package.class.
[error] bad symbolic reference. A signature in package.class refers to type Future
[error] in package akka.dispatch which is not available.
[error] It may be completely missing from the current classpath, or the version on
[error] the classpath might be incompatible with the version used when
[error] compiling package.class.
[error] two errors found
[error]
(compile:compile) Compilation failed
Do I have to add the scala library to the sbt build file as well or am I missing something else?
tld;dr: try replacing 1.0-M3 with 1.1-M7
It looks like the Spray artifacts you're using were compiled against Scala 2.9.2, which is not binary compatible with Scala 2.10. Usually, Scala artifacts contain the Scala binary version in the artifact name, precisely because major Scala versions are not binary compatible.
Later spray-util milestones were compiled against Scala 2.10.0-RC5 (based on the pom), which technically should work.
Solved by using all the same spray versions (1.1-M7) and upgrading to Scala 2.10.1