I'm trying to install Kafka in my sbt, but when I click on "import changes" I'm getting an error:
[error] stack trace is suppressed; run 'last update' for the full
output [error] stack trace is suppressed; run 'last
ssExtractDependencies' for the full output [error] (update)
sbt.librarymanagement.ResolveException: Error downloading
net.cakesolutions:scala-kafka-client_2.13:2.3.1 [error] Not found
[error] Not found [error] not found:
C:\Users\macca.ivy2\local\net.cakesolutions\scala-kafka-client_2.13\2.3.1\ivys\ivy.xml
[error] not found:
https://repo1.maven.org/maven2/net/cakesolutions/scala-kafka-client_2.13/2.3.1/scala-kafka-client_2.13-2.3.1.pom
[error] (ssExtractDependencies)
sbt.librarymanagement.ResolveException: Error downloading
net.cakesolutions:scala-kafka-client_2.13:2.3.1 [error] Not found
[error] Not found [error] not found:
C:\Users\macca.ivy2\local\net.cakesolutions\scala-kafka-client_2.13\2.3.1\ivys\ivy.xml
[error] not found:
https://repo1.maven.org/maven2/net/cakesolutions/scala-kafka-client_2.13/2.3.1/scala-kafka-client_2.13-2.3.1.pom
[error] Total time: 1 s, completed 19:56:34 26/04/2020 [info] shutting
down sbt server
build.sbt:
name := "KafkaProducer"
version := "0.1"
scalaVersion := "2.13.0"
libraryDependencies ++= Seq(
"io.circe" %% "circe-parser" % "0.12.3",
"net.cakesolutions" %% "scala-kafka-client" % "2.3.1"
)
Per the github page for scala-kafka-client, you'll need to add a bintray resolver to your build.sbt:
resolvers += Resolver.bintrayRepo("cakesolutions", "maven")
As of today Scala is still not binary compatible between versions and has tendency to serious breaking changes between "minor" (2.10 -> 2.11 -> 2.12 -> 2.13) releases.
It leads to situation where maintainers are relatively slow in adopting new versions.
e.g. Apache Spark barely started supporting 2.12 in the last stable version.
And even to the point where it is a default one.
So if I want to run this with 2.13 I have three options:
sbt publish-local
Using standard Java client instead
Nagging maintainer of Scala package to publish artificats
But I've decided to solve it by just downgrading Scala to 2.12
Related
I have a scala project using spark libraies, and it works fine most of the times(using intellij). But some times it starts giving errors on intellij launch:
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.apache.spark:spark-core_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-core_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-core_2.12\3.0.0-preview2\.spark-core_2.12-3.0.0-preview2.pom__sha1
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-core_2.12\3.0.0-preview2\.spark-core_2.12-3.0.0-preview2.pom__sha1
[error] Error downloading org.apache.spark:spark-sql_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-sql_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-sql_2.12\3.0.0-preview2\.spark-sql_2.12-3.0.0-preview2.pom__sha1
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.spark:spark-core_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-core_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-core_2.12\3.0.0-preview2\.spark-core_2.12-3.0.0-preview2.pom__sha1
[error] Error downloading org.apache.spark:spark-sql_2.12:3.0.0-preview2
[error] Not found
[error] Not found
[error] not found: C:\...\.ivy2\localorg.apache.spark\spark-sql_2.12\3.0.0-preview2\ivys\ivy.xml
[error] checksum format error: C:\....\AppData\Local\Coursier\Cache\v1\https\repo1.maven.org\maven2\org\apache\spark\spark-sql_2.12\3.0.0-preview2\.spark-sql_2.12-3.0.0-preview2.pom__sha1
[error] Total time: 1 s, completed 17 Sep 2022, 15:13:49
[info] shutting down sbt server
build.sbt is:
/*ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / scalaVersion := "2.13.8"
lazy val root = (project in file("."))
.settings(
name := "spark-learning"
)*/
// Name of the package
name := "spark-learning"
// Version of our package
version := "1.0"
// Version of Scala
scalaVersion := "2.12.14"
// Spark library dependencies
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "3.0.0-preview2",
"org.apache.spark" %% "spark-sql" % "3.0.0-preview2"
)
What causes these issues out of sudden?And how can I get rid?
One weird thing in the error messages is the path where it looks for dependencies: ...ivy2\localorg.apache.spark\....
There should be a \ after local.
Any chance your SBT repositories configuration could be messed up? Not sure where it is on Windows, it's /etc/sbt/repositories typically on Linux but maybe Intellij has its own setting.
I am studying chisel3 with a small trial project.
I finished code, fixed several syntax issues in compilation, then, it reported an error without indicating error file and line number.
$ sbt test
[info] welcome to sbt 1.4.9 (Red Hat, Inc. Java 1.8.0_292)
[info] loading settings for project fparser-build from plugins.sbt ...
[info] loading project definition from /mnt/disk1/yupeng/repos/fparser/project
[info] loading settings for project root from build.sbt ...
[info] set current project to fparser (in build file:/mnt/disk1/yupeng/repos/fparser/)
[info] compiling 3 Scala sources to /mnt/disk1/yupeng/repos/fparser/target/scala-2.12/classes ...
[error] ## Exception when compiling 3 sources to /mnt/disk1/yupeng/repos/fparser/target/scala-2.12/classes
[error] scala.reflect.internal.Types$TypeError: object plugin is not a member of package chisel3.internal
[error]
[error]
[error] scala.reflect.internal.Types$TypeError: object plugin is not a member of package chisel3.internal
[error] (Compile / compileIncremental) scala.reflect.internal.Types$TypeError: object plugin is not a member of package chisel3.internal
[error] Total time: 3 s, completed Jul 16, 2021 4:38:42 PM
what does it mean? please help.
I just found, the error is gone after I changed chisel3 versions in build.sbt.
libraryDependencies ++= Seq(
"edu.berkeley.cs" %% "chisel3" % "3.4.3",
// "edu.berkeley.cs" %% "chisel3" % "3.2.6", // this one generate plugin error above
"edu.berkeley.cs" %% "chiseltest" % "0.3.3" % "test",
"edu.berkeley.cs" %% "rocketchip" % "1.2.6"
Previously I changed from 3.4.3 to 3.2.6 because sbt warning of
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
I ran sbt evicted and it said rocketchip 1.2.6 should use chisel3 3.2.6.
Maybe someone can clarify.
I have the following sbt file
lazy val root = (project in file(".")).
settings(
inThisBuild(List(
sbtPlugin := true,
organization := "com.foo",
crossScalaVersions := Seq("2.11.2", "2.12.0"),
version := "1.0.0"
)),
name := "myplugin",
libraryDependencies ++= Seq(
"org.scala-lang.modules" %% "scala-xml" % "1.0.6",
"com.typesafe" % "config" % "1.3.3"
)
)
Now I can easily do sbt publishLocal and I see that it generates a jar file in the .ivy2/local/com.foo/myplugin/scala_2.12/sbt_1.0/1.0.0/jars/
but if I do a
sbt +publishLocal
I get an error
[error] Modules were resolved with conflicting cross-version suffixes in ProjectRef(uri("file:/Users/user/myplugin/"), "root"):
[error] org.scala-lang.modules:scala-xml _2.11, _2.12
[error] org.scala-lang.modules:scala-parser-combinators _2.11, _2.12
[error] java.lang.RuntimeException: Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml, org.scala-lang.modules:scala-parser-combinators
[error] at scala.sys.package$.error(package.scala:27)
[error] at sbt.librarymanagement.ConflictWarning$.processCrossVersioned(ConflictWarning.scala:39)
[error] at sbt.librarymanagement.ConflictWarning$.apply(ConflictWarning.scala:19)
[error] at sbt.Classpaths$.$anonfun$ivyBaseSettings$64(Defaults.scala:1995)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:39)
[error] at sbt.std.Transform$$anon$4.work(System.scala:66)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:262)
My expectation was that SBT will do the compilation and publishing twice and each time it will pick the right jars for the right Scala version. Why does it say that there is a conflict?
My end goal is to make SBT publish multiple jar files one for each scala version in my crossScalaVersions list.
sbt uses a fixed version of Scala: sbt 0.13 uses Scala 2.10, sbt 1.x uses Scala 2.12. So
one thing is that you cannot compile an sbt plugin for Scala 2.11,
and another thing is that you should cross-build your plugin for different versions of sbt (because Scala versions are implied).
There is documentation about Cross building plugins, but I'm not sure it is up to date, so it's better to see some examples in existing plugins. I think it should be enough to have this setup for your plugin project:
in project/build.properties:
sbt.version=0.13.17
in build.sbt settings:
sbtPlugin := true,
crossSbtVersions := Seq("0.13.17", "1.0.0"),
See sbt-boilerplate for an example.
After poking around with your build.sbt I found out the following:
removing all dependencies does not resolve the problem
removing sbtPlugin := true do resolve the problem
downgrading sbt from 1.x to 0.13.16 helps me compile your example
Additionally we should remember that sbt 0.13.x was written with Scala 2.10.
So, if you are writing a sbt plugin:
downgrade sbt
change Scala from 2.11.2 to 2.10.x
if you are not writing a plugin:
remove sbtPlugin := true
I am getting the following error log when setting up a scala project in intellij:
Error:Error while importing SBT project:<br/>...<br/><pre>[error] at
sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:107)
[error] at sbt.io.Using.apply(Using.scala:22)
[error] at sbt.MainLoop$.runWithNewLog(MainLoop.scala:101)
[error] at sbt.MainLoop$.runAndClearLast(MainLoop.scala:57)
[error] at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:42)
[error] at sbt.MainLoop$.runLogged(MainLoop.scala:34)
[error] at sbt.StandardMain$.runManaged(Main.scala:113)
[error] at sbt.xMain.run(Main.scala:76)
[error] at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
[error] at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
[error] at xsbt.boot.Launch$.run(Launch.scala:109)
[error] at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
[error] at xsbt.boot.Launch$.launch(Launch.scala:117)
[error] at xsbt.boot.Launch$.apply(Launch.scala:18)
[error] at xsbt.boot.Boot$.runImpl(Boot.scala:41)
[error] at xsbt.boot.Boot$.main(Boot.scala:17)
[error] at xsbt.boot.Boot.main(Boot.scala)
[error] java.lang.ClassNotFoundException: org.jetbrains.sbt.CreateTasks$
[error] Use 'last' for the full log.
[info] shutting down server</pre><br/>See complete log in file:/home/xxxx/.IdeaIC2017.2/system/log/sbt.last.log
My build.sbt looks as follows:
name := "someProjectName"
version := "0.1"
scalaVersion := "2.12.4"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.1" % "test"
Please also note that my "old" projects still work, i.e. this issue occurcs only when doing a new project. Further, I noticed that when I click on SBT projects on the very right, it opens an empty window "SBT projects", for my other projects that window is non-empty.
Any idea what I am doing wrong here?
Many thanks
c
In case you don't want the hassle of upgrading Intellij, you can downgrade the sbt version in sbt.properties file from 1.1.0 (the project default) to, for instance, 1.0.3.
I have been experiencing this problem myself, and solved it by upgrading to intellij version: 2017.3.2 build #IU-173.4127.27.
I think this may have to do with sbt 1.X series no longer being available on the typesafe bintray repository, as I was also having a problem performing some Docker builds.
I had the same problem, so I could not create any Scala class in my project. Found a workaround, by creating a scala module in my project:
Select your project folder, go to File menu, File->New->Module.
In module select Scala and setup your Scala SDK there.
Then you can create scala classes in this module.
downgrade the sbt version in sbt.properties and remove target folder from project and project/project
I have been following this tutorial to learn how to use Akka HTTP with Scala. I have no prior experience with Scala. I'm using IntelliJ Idea 2016.3 Ultimate.
I created a project and configured as the guide says.
name := "My Project"
version := "1.0"
scalaVersion := "2.9.1"
resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
libraryDependencies += "com.typesafe.akka" % "akka-actor" % "2.0"
I have the exactly same code as in the guide but when I run the SBT command compile I get the following error
[info] Updating {file:/Users/Javyer/Testing/Akka-Test/}akka-test...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/Javyer/Testing/Akka-Test/target/scala-2.9.1/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.9.1.final. Compiling...
error: error while loading CharSequence, class file '/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/rt.jar(java/lang/CharSequence.class)' is broken
(bad constant pool tag 18 at byte 10)
error: error while loading AnnotatedElement, class file '/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/rt.jar(java/lang/reflect/AnnotatedElement.class)' is broken
(bad constant pool tag 18 at byte 76)
error: error while loading Arrays, class file '/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/rt.jar(java/util/Arrays.class)' is broken
(bad constant pool tag 18 at byte 765)
error: error while loading Comparator, class file '/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/rt.jar(java/util/Comparator.class)' is broken
(bad constant pool tag 18 at byte 20)
/var/folders/kw/80c0kgzs0b9d06vjrbx0l6g40000gn/T/sbt_139df807/xsbt/ExtractAPI.scala:549: error: java.util.Comparator does not take type parameters
private[this] val sortClasses = new Comparator[Symbol] {
^
5 errors found
[info] Resolving org.scala-sbt#interface;0.13.13 ...
[error] (compile:compileIncremental) Error compiling sbt component 'compiler-interface'
[error] Total time: 5 s, completed Mar 22, 2017 11:42:21 AM
If I change my build.sbt to a more recent version of the akka libraries
name := "Akka-Test"
version := "1.0"
scalaVersion := "2.12.1"
resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.4.17",
"com.typesafe.akka" %% "akka-remote" % "2.4.17"
)
and compile again I get a different error
[info] Updating {file:/Users/Javyer/Testing/Akka-Test/}akka-test...
[info] Resolving jline#jline;2.14.1 ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/Javyer/Testing/Akka-Test/target/scala-2.12/classes...
[error] /Users/Javyer/Testing/Akka-Test/src/main/scala/Pi.scala:5: object RoundRobinRouter is not a member of package akka.routing
[error] import akka.routing.RoundRobinRouter
[error] ^
[error] /Users/Javyer/Testing/Akka-Test/src/main/scala/Pi.scala:6: object Duration is not a member of package akka.util
[error] import akka.util.Duration
[error] ^
[error] /Users/Javyer/Testing/Akka-Test/src/main/scala/Pi.scala:7: object duration is not a member of package akka.util
[error] import akka.util.duration._
[error] ^
[error] /Users/Javyer/Testing/Akka-Test/src/main/scala/Pi.scala:17: not found: type Duration
[error] case class PiApproximation(pi: Double, duration: Duration)
[error] ^
[error] /Users/Javyer/Testing/Akka-Test/src/main/scala/Pi.scala:40: not found: value RoundRobinRouter
[error] val workerRouter = context.actorOf(Props[Worker].withRouter(RoundRobinRouter(nrOfWorkers)), name = "workerRouter")
[error] ^
[error] 5 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 4 s, completed Mar 22, 2017 11:47:47 AM
How can I fix this? I don't know how to debug this error nor where to start tackling this problem. As I said, this is my first test with Scala so I don't have any prior experience with it, so, please, give me detail answers.
Thanks!
First error: Scala 2.9.x requires JVM runtime v 1.6 or 1.7, while you are using 1.8, which wouldn't work as we can clearly see from the message ('broken class' means that the runtime doesn't understand the structure of the files you are feeding it).
Second error requires a bit more context, could you post the code? I'm 99% sure you follow some outdated example, but use fresh library releases. E.g. there's no Duration in akka.util, no RoundRobinRouter in akka.routing etc.
I would suggest you to spend some time exploring/tweaking available samples, e.g. this one: https://github.com/akka/akka-samples/tree/master/akka-sample-main-scala.
Also, note that akka evolves pretty fast, but has a solid documentation versioning support, so for each library release there's a separate documentation section on the site. You'll be notified about this at leas once a day when you visit an outdated release documentation page. Here's the link to he 2.4.17 docs: http://doc.akka.io/docs/akka/2.4.17/scala.html
There are 2 options:
Using java 7 with scala 2.7.2
Using another version of scala (2.11.4), and java 8:
Example build.sbt
scalaVersion := Option(System.getProperty("scala.version")).getOrElse("2.11.4")