I'm taking the coursera scala course. I downloaded the sample project, but I can not compile it. Giving me error when I run the console command.
build.sbt:
name := course.value + "-" + assignment.value
scalaVersion := "2.12.4"
scalacOptions ++= Seq("-deprecation")
// grading libraries
libraryDependencies += "junit" % "junit" % "4.10" % Test
// for funsets
libraryDependencies += "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.4"
resolvers += "Artima Maven Repository" at "http://repo.artima.com/releases"
// include the common dir
commonSourcePackages += "common"
courseId := "bRPXgjY9EeW6RApRXdjJPw"
Error:
> console
[info] Updating root
[info] Resolved root dependencies
[trace] Stack trace suppressed: run last *:coursierResolution for the full output.
[error] (*:coursierResolution) coursier.ResolutionException: Encountered 1 error(s) in dependency resolution:
[error] org.scalatest:scalatest_2.12:2.2.4:
[error] not found:
[error] /Users/joaonobre/.ivy2/local/org.scalatest/scalatest_2.12/2.2.4/ivys/ivy.xml
[error] /Users/joaonobre/.sbt/preloaded/org.scalatest/scalatest_2.12/2.2.4/ivys/ivy.xml
[error] /Users/joaonobre/.sbt/preloaded/org/scalatest/scalatest_2.12/2.2.4/scalatest_2.12-2.2.4.pom
[error] https://repo1.maven.org/maven2/org/scalatest/scalatest_2.12/2.2.4/scalatest_2.12-2.2.4.pom
[error] http://repo.artima.com/releases/org/scalatest/scalatest_2.12/2.2.4/scalatest_2.12-2.2.4.pom
[error] Total time: 1 s, completed Dec 22, 2017 4:16:17 PM
scala -version
Scala code runner version 2.12.4 -- Copyright 2002-2017, LAMP/EPFL and Lightbend, Inc.
Any idea how to fix it?
Thanks.
As suggested by #laughedelic, Scala 2.12 does not have scalatest-2.2.4. As a result, sbt cannot find the related scalatest pom file. You can pick any of the versions mentioned here.
For instance, try changing scalaTestDependency version to 3.0.5 in the CommonBuild.scala file
lazy val scalaTestDependency = "org.scalatest" %% "scalatest" % "3.0.5"
Here is the course link (Practice Programming Assignment of Week1)
Related
I'm compiling a multi-part Scala project. It's not that large, but some of it is Scala 2.13 and some is Scala 3.
Attempting to compile generates the fatal error [UNRESOLVED DEPENDENCIES:
base#base_2.12;0.1.0-SNAPSHOT: not found]
The thing is, the string {0.1.0-SNAPSHOT} doesn't occur anywhere in my build.sbt or anywhere else. It used to be there, but it's long gone. I assume some update cache contains it, but I've been unable to find it.
Here is my {build.sbt}:
ThisBuild / libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.7" % Test
ThisBuild / Compile / scalacOptions ++= Seq("--deprecation")
ThisBuild / Test / logBuffered := false
ThisBuild / Test / parallelExecution := false
lazy val scala213 = "2.13.5"
lazy val scala212 = "2.12.13"
lazy val scala3 = "3.0.0-RC2"
lazy val supportedScalaVersions = List(scala213, scala3)
lazy val root = (project in file("."))
.aggregate(top, trans, base)
.settings(
name := "toysat"
)
lazy val top = (project in file("top"))
.settings(
name := "main",
scalaVersion := scala213,
scalacOptions += "-Ytasty-reader",
libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.7" % Test
)
.dependsOn(trans, base)
lazy val trans = (project in file("trans"))
.settings(
name := "trans",
Compile / scalaVersion := scala3,
libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.7" % Test
).
dependsOn(base)
lazy val base = (project in file("base"))
.settings(
name := "base",
scalaVersion := scala213,
libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.7" % Test
Most questions of this ilk on stackoverflow are about downloading remotely defined modules. The problem I'm having is that sbt cannot find an obsolete version of one of my (freshly compiled) modules.
and here is the sbt command output (this is an Emacs buffer):
sbt:toysat> reload
[info] welcome to sbt 1.5.5 (AdoptOpenJDK Java 1.8.0_292)
[info] loading project definition from /Users/drewmcdermott/BIG/RESEARCH/puzzles/toystory4/toysat/project
[info] loading settings for project root from build.sbt ...
[info] set current project to toysat (in build file:/Users/drewmcdermott/BIG/RESEARCH/puzzles/toystory4/toysat/)
sbt:toysat> compile
[info] compiling 4 Scala sources to /Users/drewmcdermott/BIG/RESEARCH/puzzles/toystory4/toysat/base/target/scala-2.13/classes ...
[warn]
[warn] Note: Unresolved dependencies path:
[info] done compiling
[error] stack trace is suppressed; run last trans / update for the full output
[error] (trans / update) sbt.librarymanagement.ResolveException: Error downloading base:base_2.12:0.1.0-SNAPSHOT
[error] Not found
[error] Not found
[error] not found: /Users/drewmcdermott/.ivy2/localbase/base_2.12/0.1.0-SNAPSHOT/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/base/base_2.12/0.1.0-SNAPSHOT/base_2.12-0.1.0-SNAPSHOT.pom
[error] Total time: 25 s, completed Jul 28, 2021 11:06:18 PM
The 25 seconds were consumed compiling the 4 files in the base subproject, apparently successfully. I think it's when sbt tries to compile the trans subproject that it runs into trouble.
Here's a partial stack trace. It means nothing to me except that Coursier is involved.
sbt:toysat> last trans / update
[debug] not up to date. inChanged = true, force = false
[debug] Updating trans...
[warn]
[warn] Note: Unresolved dependencies path:
[error] sbt.librarymanagement.ResolveException: Error downloading base:base_2.12:0.1.0-SNAPSHOT
[error] Not found
[error] Not found
[error] not found: /Users/drewmcdermott/.ivy2/localbase/base_2.12/0.1.0-SNAPSHOT/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/base/base_2.12/0.1.0-SNAPSHOT/base_2.12-0.1.0-SNAPSHOT.pom
[error] at lmcoursier.CoursierDependencyResolution.unresolvedWarningOrThrow(CoursierDependencyResolution.scala:258)
[error] at lmcoursier.CoursierDependencyResolution.$anonfun$update$38(CoursierDependencyResolution.scala:227)
[error] at lmcoursier.CoursierDependencyResolution$$Lambda$4262/0x0000000000000000.apply(Unknown Source)
[error] at scala.util.Either$LeftProjection.map(Either.scala:573)
[error] at lmcoursier.CoursierDependencyResolution.update(CoursierDependencyResolution.scala:227)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:60)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:59)
It seems clear that some cache somewhere is holding onto the string 0.1.0-SNAPSHOT, but there are an ungodly number of caches. I've tried deleting several, but I haven't found the relevant one.
Can someone explain how to recover from a situation like this?
Your base project is only compiled for Scala 2.13 whereas it is defined as a dependency (using dependsOn) of trans which targets Scala 3.
You should cross-build your base project for Scala 2.13 and 3 (and maybe 2.12 according to your error message even though I don't see any use of Scala 2.12 in what you shared).
Edit: Scala 2.13 and 3 are compatible, so the issue should only happen if a dependency is built only for 2.12.
I am not answering my own question because I'm a narcissist, but because I can't say what I want in a comment. Plus editing the original question would bury possibly useful information in an odd place. I've upvoted and approved #GaelJ's answer.
My build.sbt doesn't look that different. The differences may be encapsulated by showing the revised trans subproject definition:
lazy val trans = (projectMatrix in file("trans"))
.settings(
name := "trans",
version := "0.3",
// I thought this line was unnecessary, but without
// it sbt doesn't understand the command trans / compile --
Compile / scalaVersion := scala3,
libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.7" % Test
)
.jvmPlatform(scalaVersions = Seq(scala213))
.dependsOn(base)
Here is a minimal reproducible example:
echo '
libraryDependencies += "org.apache.jena" % "apache-jena-libs" % "3.17.0" ;
lazy val testPlay = (project in file(".")) .enablePlugins(PlayScala)
' > build.sbt ;
mkdir project ;
echo 'addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.8.5")'
> project/plugins.sbt ;
sbt run
I have a dependency to a Java module (Apache Jena), that specifies jackson-core 2.11.3 , while latest Play is at 2.10.4 . Play does not start in dev mode:
JsonMappingException: Scala module 2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0.
I tried this:
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.10.4"
but it is not taken in account, checking with show fullClasspath and dependencyTree .
Also tried excludeDependencies in two ways, specifying 2.10.4 or 2.11.3 . My understanding is that
a Java dependency cannot be changed by SBT excludeDependencies
function play.runsupport.Reloader.startDevMode() is not
affected by main SBT configuration, because it is part of the Play
plugin ; indeed the class Reloader is not in play-server_2.12-2.8.5.jar
So what kind of hack could I try?
The sbt config.
https://github.com/jmvanel/semantic_forms/blob/master/scala/project/Common.scala#L44
The stack trace:
[error] com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.10.4 requires Jackson Databind version >= 2.10.0 and < 2.11.0
[error] at com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:61)
[error] at com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:46)
[error] at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:17)
[error] at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:819)
[error] at akka.serialization.jackson.JacksonObjectMapperProvider$.$anonfun$configureObjectMapperModules$4(JacksonObjectMapperProvider.scala:223)
[error] at akka.serialization.jackson.JacksonObjectMapperProvider$.$anonfun$configureObjectMapperModules$4$adapted(JacksonObjectMapperProvider.scala:222)
[error] at scala.collection.immutable.List.foreach(List.scala:431)
[error] at akka.serialization.jackson.JacksonObjectMapperProvider$.configureObjectMapperModules(JacksonObjectMapperProvider.scala:222)
...
[error] at akka.actor.ActorSystem$.apply(ActorSystem.scala:290)
[error] at play.core.server.DevServerStart$.$anonfun$mainDev$1(DevServerStart.scala:248)
[error] at play.utils.Threads$.withContextClassLoader(Threads.scala:22)
[error] at play.core.server.DevServerStart$.mainDev(DevServerStart.scala:76)
You can force a dependency with:
libraryDependencies += "com.fasterxml.jackson.core" % "jackson-databind" % "2.10.4" force()
Another option you have is to exclude dependency:
libraryDependencies += "org.apache.jena" % "apache-jena-libs" % "3.17.0" exclude("com.fasterxml.jackson.core", "jackson-databind")
I am new to scala and I am trying to import the following libraries in my build.sbt. When IntelliJ does an auto-update I get the following error:
Error while importing sbt project:
List([info] welcome to sbt 1.3.13 (Oracle Corporation Java 1.8.0_251)
[info] loading global plugins from C:\Users\diego\.sbt\1.0\plugins
[info] loading project definition from C:\Users\diego\development\Meetup\Stream-Processing\project
[info] loading settings for project stream-processing from build.sbt ...
[info] set current project to Stream-Processing (in build file:/C:/Users/diego/development/Meetup/Stream-Processing/)
[info] sbt server started at local:sbt-server-80d70f9339b81b4d026a
sbt:Stream-Processing>
[info] Defining Global / sbtStructureOptions, Global / sbtStructureOutputFile and 1 others.
[info] The new values will be used by cleanKeepGlobs
[info] Run `last` for details.
[info] Reapplying settings...
[info] set current project to Stream-Processing (in build file:/C:/Users/diego/development/Meetup/Stream-Processing/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from C:/Users/diego/.IntelliJIdea2019.3/config/plugins/Scala/repo/org.jetbrains/sbt-structure-extractor/scala_2.12/sbt_1.0/2018.2.1+4-88400d3f/jars/sbt-structure-extractor.jar
[info] Reapplying settings...
[info] set current project to Stream-Processing (in build file:/C:/Users/diego/development/Meetup/Stream-Processing/)
[warn]
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.apache.kafka:kafka-clients_2.11:2.3.1
[error] Not found
[error] Not found
[error] not found: C:\Users\diego\.ivy2\local\org.apache.kafka\kafka-clients_2.11\2.3.1\ivys\ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/kafka/kafka-clients_2.11/2.3.1/kafka-clients_2.11-2.3.1.pom
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.kafka:kafka-clients_2.11:2.3.1
[error] Not found
[error] Not found
[error] not found: C:\Users\diego\.ivy2\local\org.apache.kafka\kafka-clients_2.11\2.3.1\ivys\ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/kafka/kafka-clients_2.11/2.3.1/kafka-clients_2.11-2.3.1.pom
[error] Total time: 2 s, completed Jun 28, 2020 12:11:24 PM
[info] shutting down sbt server)
This is my build.sbt file:
name := "Stream-Processing"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.4"
// https://mvnrepository.com/artifact/org.apache.spark/spark-sql-kafka-0-10_2.12
libraryDependencies += "org.apache.spark" %% "spark-sql-kafka-0-10" % "2.4.4"
// https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients
libraryDependencies += "org.apache.kafka" %% "kafka-clients" % "2.3.1"
// https://mvnrepository.com/artifact/mysql/mysql-connector-java
libraryDependencies += "mysql" % "mysql-connector-java" % "8.0.18"
// https://mvnrepository.com/artifact/org.mongodb.spark/mongo-spark-connector
libraryDependencies += "org.mongodb.spark" %% "mongo-spark-connector" % "2.4.1"
I made a Scala project just to make sure Spark works and my python project using Kafka works as well so I am sure it's not a spark/kafka problem. Any reason why I am getting that error?
Try removing one % before "kafka-clients":
libraryDependencies += "org.apache.kafka" % "kafka-clients" % "2.3.1"
The semantics of %% in SBT is that it appends the Scala version being used to the artifact name, so it becomes org.apache.kafka:kafka-clients_2.11:2.3.1 as the error message shows as well. Note the _2.11 suffix.
This is a nice shorthand for Scala libraries, but can get confusing for beginners, when used with Java libs.
I have installed Flink, Scala and sbt
Flink Version: 1.9.1
Scala Version: 2.10.6
Sbt Version: 1.3.7
I made relevant changes in build.sbt.
Compile command is failing
Here is the relevant information.
Any information is greatly appreciated
**Versions Information
[osboxes#osboxes local]$ scala -version
Scala code runner version 2.10.6 -- Copyright 2002-2013, LAMP/EPFL
[osboxes#osboxes local]$ flink --version
Version: 1.9.1, Commit ID: 4d56de8
[osboxes#osboxes readcsvfile]$ sbt -version
sbt version in this project: 1.3.7
sbt script version: 1.3.7
** build.sbt changes
val flinkVersion = "1.9.1"
val flinkDependencies = Seq(
"org.apache.flink" %% "flink-scala" % flinkVersion % "provided",
"org.apache.flink" %% "flink-streaming-scala" % flinkVersion % "provided")
** Compile Errors
sbt:readCsvfile> compile
[info] Updating
[info] Resolved dependencies
[warn]
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run last update for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.apache.flink:flink-streaming-scala_2.13:1.9.1
[error] Not found
[error] Not found
[error] not found: /home/osboxes/.ivy2/local/org.apache.flink/flink-streaming-scala_2.13/1.9.1/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/flink/flink-streaming-scala_2.13/1.9.1/flink-streaming-scala_2.13-1.9.1.pom
[error] Error downloading org.apache.flink:flink-scala_2.13:1.9.1
[error] Not found
[error] Not found
[error] not found: /home/osboxes/.ivy2/local/org.apache.flink/flink-scala_2.13/1.9.1/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/flink/flink-scala_2.13/1.9.1/flink-scala_2.13-1.9.1.pom
[error] Total time: 4 s, completed Jan 30, 2020 3:59:12 PM
sbt:readCsvfile>
Few points I want to mention here regarding the SBT dependencies issues are:
Please add scalaVersion := "2.12.11" in build.sbt file like this, which includes the Scala version in your SBT dependencies automatically due to this%%.
name := "flink-streaming-demo"
scalaVersion := "2.12.11"
val flinkVersion = "1.10.0"
libraryDependencies += "org.apache.flink" %% "flink-scala" % flinkVersion % "provided"
libraryDependencies += "org.apache.flink" %% "flink-streaming-scala" % flinkVersion % "provided"
If you want Scala version specific SBT dependencies then use % like this:
libraryDependencies += "org.apache.flink" % "flink-scala_2.12" % flinkVersion % "provided"
libraryDependencies += "org.apache.flink" % "flink-streaming-scala_2.12" % flinkVersion % "provided"
In worst case if all these does not work then simply delete or rename these existing .sbt and .ivy2 hidden folder in your system home directory, where your all dependecies and plugins get sotred after downloading from maven central and then refresh/build the SBT project.
SBT dependency format
libraryDependencies += groupID % artifactID % revision % configuration
Meaning of % and %%
%: A method used to construct an Ivy Module ID from the strings you supply.
%%: When used after the groupID, it automatically adds your project’s Scala version (such as _2.12) to the end of the artifact name.
NOTE: To get more details click here.
summing up the comments since perhaps it is a bit hard to know what you should do
In general, if you get an "Unresolved dependencies" error, look at mvnrepository.com, search for your artifact:
https://mvnrepository.com/artifact/org.apache.flink/flink-scala
This tells you (second column) which Scala versions are supported by it. In this case, the library is available for 2.11.x and 2.12.x.
Thus, you have to use a Scala version compatible with that in your build, in build.sbt:
ThisBuild / scalaVersion := "2.12.10"
I am trying the add the mockito dependency in the SBT build file, But it gives below unresolved dependency exception.
I am not sure if its the issue with Scala and Mockito version.
scalaVersion := "2.11.8"
libraryDependencies ++= Seq("org.scalatest" % "scalatest_2.11" % "2.2.2" % "test",
"org.mockito" % "mockito-all" % "1.9.5" % "test")
Exception :
trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.mockito#mockito-all;1.9.5: not found
[error] (*:update) sbt.ResolveException: unresolved dependency: org.mockito#mockito-all;1.9.5: not found
[error] Total time: 23 s, completed Oct 12, 2017 3:13:16 PM
I have tried with different mockito version 1.8.5 as well. But no luck.
Thanks in advance.
I strongly advise not to use mockito-all with sbt, but use mockito-core instead. SBT is smart build system and it will figure out all dependencies for mockito-core, details. Here how your dependencies might look:
libraryDependencies ++= Seq (
"org.scalatest" %% "scalatest" % "3.0.1" % "test",
"org.mockito" % "mockito-core" % "2.8.47" % "test"
)
The Mockito documentation advises to not use mockito-core as a dependency, but to use mockito-scala which will pull what's needed. You could use the %% notation which automatically pulls the version appropriate for the Scala version you are using. It's also worth looking on Maven to see which is the latest mockito-scala version for the Scala version you are using.
So the build.sbt for Mockito used with Scala with today's latest version would be:
libraryDependencies ++= Seq (
"org.mockito" %% "mockito-scala" % "1.15.0" % "test"
)