sbt can not compile Scala project because of java.lang.NoSuchMethodError - scala

Question

I am new to Scala so any help would be really appreciated.
I am using IntelliJ IDEA Version: 2020.1, sbt.version=1.2.8, jdk1.8.0_ 251 and Scala 2.12.8.
When I tried to compile a coursera project i get the following error
compile
[error] stack trace is suppressed; run 'last coursierResolutions' for the full output
[error] (coursierResolutions) java.lang.NoSuchMethodError: lmcoursier.definitions.ToCoursier$.project(Llmcoursier/definitions/Project;)Lcoursier/core/Project;
[error] Total time: 0 s, completed 23-Apr-2020 23:46:04
[IJ]sbt:bigdata-wikipedia> last coursierResolutions
[error] java.lang.NoSuchMethodError: lmcoursier.definitions.ToCoursier$.project(Llmcoursier/definitions/Project;)Lcoursier/core/Project;
[error] at coursier.sbtcoursier.ResolutionTasks$.$anonfun$resolutionsTask$3(ResolutionTasks.scala:43)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:281)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:19)
[error] at sbt.Execute.work(Execute.scala:290)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:281)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
This is the sbt coursier plugin in plugins.sbt
addSbtPlugin("io.get-coursier" % "sbt-coursier" % "2.0.0-RC3-5")
addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.9.7")

Answer 1

Removing the coursier plugin and upgrading to sbt 1.3.13 worked for me.
build.properties
sbt.version=1.3.13
plugins.sbt
//addSbtPlugin("io.get-coursier" % "sbt-coursier" % "2.0.0-RC3-5")
addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.9.7")
build.sbt
course := "bigdata"
assignment := "wikipedia"
build.sbt
scalaVersion := "2.12.11"
scalacOptions ++= Seq("-language:implicitConversions", "-deprecation")
libraryDependencies ++= Seq(
"com.novocode" % "junit-interface" % "0.11" % Test,
("org.apache.spark" %% "spark-core" % "2.4.3"),
("org.apache.spark" %% "spark-sql" % "2.4.3")
)
dependencyOverrides ++= Seq(
("com.fasterxml.jackson.core" % "jackson-databind" % "2.6.7")
)
testOptions in Test += Tests.Argument(TestFrameworks.JUnit, "-a", "-v", "-s")
Credit to #JOHN for this answer.

Answer 2

I used the sbt binary in terminal directly with version 1.2.8 and it saved me(the sbt version of sbt shell in IDEA was 1.3.8).

Answer 3

I was experiencing the same error while taking the course Functional Programming Principles in Scala by École Polytechnique Fédérale de Lausanne on Coursera (I assume you were taking that course too). For me the solution was using the following settings, where my changes were in the "sbt projects" section at the bottom:
After that, see how from sbt shell everything worked correctly for me:
"C:\Program Files\AdoptOpenJDK\jdk-11.0.9.11-hotspot\bin\java.exe" -agentlib:jdwp=transport=dt_socket,address=localhost:63820,suspend=n,server=y -Xdebug -server -Xmx1536M -Dsbt.supershell=false -Didea.managed=true -Dfile.encoding=UTF-8 -Dsbt.log.noformat=true -jar C:\Users\jaime\AppData\Roaming\JetBrains\IdeaIC2020.2\plugins\Scala\launcher\sbt-launch.jar early(addPluginSbtFile=\"\"\"C:\Users\jaime\AppData\Local\Temp\idea1.sbt\"\"\") "; set ideaPort in Global := 63650 ; idea-shell"
Listening for transport dt_socket at address: 63820
[info] Loading settings for project global-plugins from idea1.sbt ...
[info] Loading global plugins from C:\Users\jaime\.sbt\1.0\plugins
[info] Updating ProjectRef(uri("file:/C:/Users/jaime/.sbt/1.0/plugins/"), "global-plugins")...
[info] downloading https://repo1.maven.org/maven2/io/github/sugakandrey/scala-compiler-indices-protocol_2.12/0.1.1/scala-compiler-indices-protocol_2.12-0.1.1.jar ...
[info] downloading https://repo1.maven.org/maven2/io/spray/spray-json_2.12/1.3.4/spray-json_2.12-1.3.4.jar ...
[info] downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.jetbrains/sbt-structure-extractor/scala_2.12/sbt_1.0/2018.2.1+4-88400d3f/jars/sbt-structure-extractor.jar ...
[info] [SUCCESSFUL ] io.github.sugakandrey#scala-compiler-indices-protocol_2.12;0.1.1!scala-compiler-indices-protocol_2.12.jar (380ms)
[info] [SUCCESSFUL ] io.spray#spray-json_2.12;1.3.4!spray-json_2.12.jar(bundle) (444ms)
[info] downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.jetbrains/sbt-idea-shell/scala_2.12/sbt_1.0/2018.3/jars/sbt-idea-shell.jar ...
[info] downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.jetbrains/sbt-idea-compiler-indices/scala_2.12/sbt_1.0/0.1.3/jars/sbt-idea-compiler-indices.jar ...
[info] [SUCCESSFUL ] org.jetbrains#sbt-structure-extractor;2018.2.1+4-88400d3f!sbt-structure-extractor.jar (1711ms)
[info] [SUCCESSFUL ] org.jetbrains#sbt-idea-shell;2018.3!sbt-idea-shell.jar (1790ms)
[info] [SUCCESSFUL ] org.jetbrains#sbt-idea-compiler-indices;0.1.3!sbt-idea-compiler-indices.jar (2273ms)
[info] Done updating.
[info] Loading settings for project example-build from buildSettings.sbt,plugins.sbt ...
[info] Loading project definition from C:\Users\jaime\Downloads\example\project
[info] Updating ProjectRef(uri("file:/C:/Users/jaime/Downloads/example/project/"), "example-build")...
[info] Done updating.
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[info] Compiling 2 Scala sources to C:\Users\jaime\Downloads\example\project\target\scala-2.12\sbt-1.0\classes ...
[info] Done compiling.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/C:/Users/jaime/.sbt/boot/scala-2.12.7/org.scala-sbt/sbt/1.2.8/protobuf-java-3.3.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[info] Loading settings for project example from assignment.sbt,build.sbt ...
[info] Set current project to progfun1-example (in build file:/C:/Users/jaime/Downloads/example/)
[info] Defining Global / ideaPort
[info] The new value will be used by Compile / compile, Test / compile
[info] Reapplying settings...
[info] Set current project to progfun1-example (in build file:/C:/Users/jaime/Downloads/example/)
[IJ]sbt:progfun1-example> console
[info] Compiling 1 Scala source to C:\Users\jaime\Downloads\example\target\scala-2.13\classes ...
[info] Non-compiled module 'compiler-bridge_2.13' for Scala 2.13.0. Compiling...
[info] Compilation completed in 7.345s.
[info] Done compiling.
[info] Starting scala interpreter...
Welcome to Scala 2.13.0 (OpenJDK 64-Bit Server VM, Java 11.0.9).
Type in expressions for evaluation. Or try :help.
scala>

Answer 4

Yes, that solution worked for me partially, but I also had to change the build.sbt as following:
course := "bigdata"
assignment := "wikipedia"
scalaVersion := "2.12.8"
scalacOptions ++= Seq("-language:implicitConversions", "-deprecation")
libraryDependencies ++= Seq(
"com.novocode" % "junit-interface" % "0.11" % Test,
("org.apache.spark" %% "spark-core" % "2.4.3"),
("org.apache.spark" %% "spark-sql" % "2.4.3")
)
dependencyOverrides ++= Seq(
("com.fasterxml.jackson.core" % "jackson-databind" % "2.6.7")
)
testOptions in Test += Tests.Argument(TestFrameworks.JUnit, "-a", "-v", "-s")
That worked for relatively new Scala version and recent Spark libraries built for it.