I am trying to automate CI/CD of a small Scala project using CircleCI. The project is built using sbt, and tested using the ScalaTest library.
As per the ScalaTest installation instruction's recommendation, I am using the SuperSafe compiler plugin, which required me to add a resolver to global file ~/.sbt/1.0/global.sbt:
resolvers += "Artima Maven Repository" at "http://repo.artima.com/releases"
I can successfully compile and test my project locally. however on CircleCI the build fails with error:
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.artima.supersafe#supersafe_2.12.8;1.1.7: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] com.artima.supersafe:supersafe_2.12.8:1.1.7 (Defaults.scala#L3331)
[warn] +- filesystem:filesystem_2.12:0.1
[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.artima.supersafe#supersafe_2.12.8;1.1.7: not found
[error] at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error] at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error] at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error] at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error] at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error] at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error] at xsbt.boot.Using$.withResource(Using.scala:10)
[error] at xsbt.boot.Using$.apply(Using.scala:9)
[error] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error] at xsbt.boot.Locks$.apply(Locks.scala:28)
[error] at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error] at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error] at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error] at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error] at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error] at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error] at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error] at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error] at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2561)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error] at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:278)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: com.artima.supersafe#supersafe_2.12.8;1.1.7: not found
This error is to be expected when the required resolver hasn't been added in SBT -- see e.g. this issue. I am new to CircleCI and don't know where its SBT globals dir would be located, or how to modify the file.
Instead I've tried to add the resolver to my project's ./project/plugins.sbt file directly, but this has not fixed the issue.
The SBT and CircleCI config files look as follows:
./build.sbt
name := "my-project-name"
version := "0.1"
scalaVersion := "2.12.8"
libraryDependencies += "org.scalactic" %% "scalactic" % "3.0.8"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.8" % "test"
./project/build.properties
sbt.version = 1.2.8
./project/plugins.sbt
resolvers += "Artima Maven Repository" at "http://repo.artima.com/releases"
addSbtPlugin("com.artima.supersafe" % "sbtplugin" % "1.1.7")
./.circleci/config.yml
(the default Scala config provided by CircleCI)
# Scala CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/sample-config/ for more details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
- image: circleci/openjdk:8-jdk
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/repo
environment:
# Customize the JVM maximum heap limit
JVM_OPTS: -Xmx3200m
TERM: dumb
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "build.sbt" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run: cat /dev/null | sbt test:compile
- save_cache:
paths:
- ~/.m2
key: v1-dependencies--{{ checksum "build.sbt" }}
# run tests!
- run: cat /dev/null | sbt test:test
I want a successful CircleCI build, which will require a way to add the resolver, either in the project's own files or declaring it in the SBT globals file of the CircleCI container.
Consider open issue SBT isn't using resolvers defined in project/plugins.sbt #4103. Try scoping the resolver to ThisBuild and put it in both build.sbt and plugins.sbt like so:
// someApp/build.sbt
resolvers in ThisBuild += "Artima Maven Repository" at "http://repo.artima.com/releases"
and
// someApp/project/plugins.sbt
resolvers in ThisBuild += "Artima Maven Repository" at "http://repo.artima.com/releases"
addSbtPlugin("com.artima.supersafe" % "sbtplugin" % "1.1.7")
This seems to have worked on this example repo
Related
It appears that SBT can't find any of the dependencies needed, as well as both blooper and sbt can't seem to connect online to download any dependencies. I have checked if I have a proxy, if there are any inconsistent versions that are conflicting, and have fully reinstalled SBT, NVM, VSCode and everything else involved. I can't seem to figure out why it won't build and why it can't find any of the dependencies.
EDIT: Added Build.sbt file
I've removed the https:// from the repo links, but they are there in the normal error.
build.sbt file:
import Dependencies._
ThisBuild / scalaVersion := "2.12.11"
ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / organization := "????"
ThisBuild / organizationName := "????"
lazy val root = (project in file("."))
.settings(
name := "ScalaJS-only",
libraryDependencies += "org.scala-js" %%% "scalajs-dom" % "1.0.0",
libraryDependencies += scalaTest % Test,
scalaJSUseMainModuleInitializer := true
).enablePlugins(ScalaJSPlugin)
Error Received:
PS C:\Users\????\Documents\Backend> sbt
[info] Loading settings for project Backend-build-build-build from metals.sbt ...
[info] Loading project definition from C:\Users\????\Documents\Backend\project\project\project
Oct 19, 2021 11:46:44 AM lmcoursier.internal.shaded.coursier.cache.shaded.org.jline.utils.Log logr
WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
[info] Updating
[info] Resolved dependencies
[warn]
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] ch.epfl.scala:sbt-bloop:1.4.9 (sbtVersion=1.0, scalaVersion=2.12)
[warn]
[warn] Note: Unresolved dependencies path:
[error] sbt.librarymanagement.ResolveException: Error downloading org.scala-sbt:sbt:1.3.5
[error] Not found
[error] Not found
[error] wrong checksum: C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-sbt\sbt\1.3.5\sbt-1.3.5.pom (expected SHA-1 88a91f2b0e535d0c22bc0b9dcc9b586bc6b7a109 in C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-sbt\sbt\1.3.5\.sbt-1.3.5.pom__sha1, got aea457bc4067a7c2080a43791cdeb78b072ff5bb)
[error] not found: C:\Users\????\.ivy2\local\org.scala-sbt\sbt\1.3.5\ivys\ivy.xml
[error] not found: repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.scala-sbt/sbt/1.3.5/ivys/ivy.xml
[error] not found: repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt/1.3.5/ivys/ivy.xml
[error] Error downloading ch.epfl.scala:sbt-bloop;sbtVersion=1.0;scalaVersion=2.12:1.4.9
[error] Not found
[error] Not found
[error] wrong checksum: C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\epfl\scala\sbt-bloop_2.12_1.0\1.4.9\sbt-bloop-1.4.9.pom (expected SHA-1 a9f21bf5ab89ca29d3acbac10c2a7ab74dbf8646 in C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\epfl\scala\sbt-bloop_2.12_1.0\1.4.9\.sbt-bloop-1.4.9.pom__sha1, got 3f431c638c39d83018ca88a348d1fb0e62db43a0)
[error] not found: C:\Users\????\.ivy2\local\ch.epfl.scala\sbt-bloop\scala_2.12\sbt_1.0\1.4.9\ivys\ivy.xml
[error] not found: repo.scala-sbt.org/scalasbt/sbt-plugin-releases/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/1.4.9/ivys/ivy.xml
[error] not found: repo.typesafe.com/typesafe/ivy-releases/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/1.4.9/ivys/ivy.xml
[error] Error downloading org.scala-lang:scala-library:2.12.10
[error] Not found
[error] Not found
[error] wrong checksum: C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.12.10\scala-library-2.12.10.pom (expected SHA-1 368a4cdefd92e20800076e97e3b58e722a7f942a in C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.12.10\.scala-library-2.12.10.pom__sha1, got c0f5237f49d4b4a955c368a646e7eebc8ca3fc84)
[error] not found: C:\Users\????\.ivy2\local\org.scala-lang\scala-library\2.12.10\ivys\ivy.xml
[error] not found: repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.scala-lang/scala-library/2.12.10/ivys/ivy.xml
[error] not found: repo.typesafe.com/typesafe/ivy-releases/org.scala-lang/scala-library/2.12.10/ivys/ivy.xml
[error] at lmcoursier.CoursierDependencyResolution.unresolvedWarningOrThrow(CoursierDependencyResolution.scala:245)
[error] at lmcoursier.CoursierDependencyResolution.$anonfun$update$34(CoursierDependencyResolution.scala:214)
[error] at scala.util.Either$LeftProjection.map(Either.scala:573)
[error] at lmcoursier.CoursierDependencyResolution.update(CoursierDependencyResolution.scala:214)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:60)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:52)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:102)
[error] at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:69)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$20(LibraryManagement.scala:115)
[error] at scala.util.control.Exception$Catch.apply(Exception.scala:228)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:115)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:96)
[error] at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:150)
[error] at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:129)
[error] at sbt.Classpaths$.$anonfun$updateTask0$5(Defaults.scala:2947)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:281)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:19)
[error] at sbt.Execute.work(Execute.scala:290)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:281)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] at java.base/java.lang.Thread.run(Thread.java:834)
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.scala-sbt:sbt:1.3.5
[error] Not found
[error] Not found
[error] wrong checksum: C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-sbt\sbt\1.3.5\sbt-1.3.5.pom (expected SHA-1 88a91f2b0e535d0c22bc0b9dcc9b586bc6b7a109 in C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-sbt\sbt\1.3.5\.sbt-1.3.5.pom__sha1, got aea457bc4067a7c2080a43791cdeb78b072ff5bb)
[error] not found: C:\Users\????\.ivy2\local\org.scala-sbt\sbt\1.3.5\ivys\ivy.xml
[error] not found: repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.scala-sbt/sbt/1.3.5/ivys/ivy.xml
[error] not found: repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt/1.3.5/ivys/ivy.xml
[error] Error downloading ch.epfl.scala:sbt-bloop;sbtVersion=1.0;scalaVersion=2.12:1.4.9
[error] Not found
[error] Not found
[error] wrong checksum: C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\epfl\scala\sbt-bloop_2.12_1.0\1.4.9\sbt-bloop-1.4.9.pom (expected SHA-1 a9f21bf5ab89ca29d3acbac10c2a7ab74dbf8646 in C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\epfl\scala\sbt-bloop_2.12_1.0\1.4.9\.sbt-bloop-1.4.9.pom__sha1, got 3f431c638c39d83018ca88a348d1fb0e62db43a0)
[error] not found: C:\Users\????\.ivy2\local\ch.epfl.scala\sbt-bloop\scala_2.12\sbt_1.0\1.4.9\ivys\ivy.xml
[error] not found: repo.scala-sbt.org/scalasbt/sbt-plugin-releases/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/1.4.9/ivys/ivy.xml
[error] not found: repo.typesafe.com/typesafe/ivy-releases/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/1.4.9/ivys/ivy.xml
[error] Error downloading org.scala-lang:scala-library:2.12.10
[error] Not found
[error] Not found
[error] wrong checksum: C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.12.10\scala-library-2.12.10.pom (expected SHA-1 368a4cdefd92e20800076e97e3b58e722a7f942a in C:\Users\????\Documents\Backend\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.12.10\.scala-library-2.12.10.pom__sha1, got c0f5237f49d4b4a955c368a646e7eebc8ca3fc84)
[error] not found: C:\Users\????\.ivy2\local\org.scala-lang\scala-library\2.12.10\ivys\ivy.xml
[error] not found: repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.scala-lang/scala-library/2.12.10/ivys/ivy.xml
[error] not found: repo.typesafe.com/typesafe/ivy-releases/org.scala-lang/scala-library/2.12.10/ivys/ivy.xml
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?
We published our sbt plugin to local m2 repository using sbt clean compile publishM2.
Below is the simplified plugin source code :-
build.sbt :-
name := "base-plugin"
version := "0.1"
organization := "io.github.moglideveloper"
sbtPlugin := true
val pluginDeveloper = Developer("mogli", "mogli", "moglideveloper#gmail.com", url("https://github.com/moglideveloper"))
developers := List(pluginDeveloper)
homepage := Some(url("https://github.com/moglideveloper/BasePlugin"))
scmInfo := Some(ScmInfo(url("https://github.com/moglideveloper/BasePlugin"), "git#github.com:moglideveloper/BasePlugin.git"))
licenses += ("MIT", url("https://opensource.org/licenses/MIT"))
publishMavenStyle := true
publishArtifact in Test := false
publishConfiguration := publishConfiguration.value.withOverwrite(true)
publishTo := Some(Opts.resolver.sonatypeStaging)
build.properties :-
sbt.version=1.2.8
We are using 1.2.8 because of existing issues of sbt upgrade :-
https://github.com/xerial/sbt-sonatype/pull/98
https://github.com/sbt/sbt/issues/5049
BasePlugin.scala :-
package io.moglideveloper.base_plugin
import sbt._
object BasePlugin extends AutoPlugin {
lazy val hello = taskKey[Unit]("just say hello")
override def trigger = allRequirements
override lazy val projectSettings = Seq(
hello := {
println("hello")
}
)
}
Now, above published plugin is not resolved correctly with any sbt version that is 1.x, for example sbt.version=1.2.8
Below is the exception trace :-
/Users/mogli/.sdkman/candidates/java/11.0.9.hs-adpt/bin/java -Djline.terminal=jline.UnsupportedTerminal -Dsbt.log.noformat=true -Dfile.encoding=UTF-8 -Didea.managed=true -Dfile.encoding=UTF-8 -jar /Users/mogli/Library/Application Support/JetBrains/IdeaIC2020.3/plugins/Scala/launcher/sbt-launch.jar
[info] Loading settings for project resolvecheck-build from plugins.sbt ...
[info] Loading project definition from /Users/mogli/sources/ResolveCheck/project
[info] Updating ProjectRef(uri("file:/Users/mogli/sources/ResolveCheck/project/"), "resolvecheck-build")...
[warn] module not found: io.github.moglideveloper#base-plugin;0.1
[warn] ==== typesafe-ivy-releases: tried
[warn] https://repo.typesafe.com/typesafe/ivy-releases/io.github.moglideveloper/base-plugin/scala_2.12/sbt_1.0/0.1/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/io.github.moglideveloper/base-plugin/scala_2.12/sbt_1.0/0.1/ivys/ivy.xml
[warn] ==== local: tried
[warn] /Users/mogli/.ivy2/local/io.github.moglideveloper/base-plugin/scala_2.12/sbt_1.0/0.1/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/io/github/moglideveloper/base-plugin_2.12_1.0/0.1/base-plugin-0.1.pom
[warn] ==== local-preloaded-ivy: tried
[warn] /Users/mogli/.sbt/preloaded/io.github.moglideveloper/base-plugin/0.1/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:////Users/mogli/.sbt/preloaded/io/github/moglideveloper/base-plugin_2.12_1.0/0.1/base-plugin-0.1.pom
[warn] ==== Local Repository: tried
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: io.github.moglideveloper#base-plugin;0.1: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] io.github.moglideveloper:base-plugin:0.1 (scalaVersion=2.12, sbtVersion=1.0)
[warn]
[warn] Note: Unresolved dependencies path:
[warn] io.github.moglideveloper:base-plugin:0.1 (scalaVersion=2.12, sbtVersion=1.0) (/Users/mogli/sources/ResolveCheck/project/plugins.sbt#L2-3)
[warn] +- default:resolvecheck-build:0.1.0-SNAPSHOT (scalaVersion=2.12, sbtVersion=1.0)
[error] sbt.librarymanagement.ResolveException: unresolved dependency: io.github.moglideveloper#base-plugin;0.1: not found
[error] at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error] at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error] at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error] at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error] at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error] at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error] at xsbt.boot.Using$.withResource(Using.scala:10)
[error] at xsbt.boot.Using$.apply(Using.scala:9)
[error] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error] at xsbt.boot.Locks$.apply(Locks.scala:28)
[error] at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error] at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error] at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error] at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error] at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error] at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error] at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error] at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error] at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2561)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error] at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:278)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] at java.base/java.lang.Thread.run(Thread.java:834)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: io.github.moglideveloper#base-plugin;0.1: not found
Invalid response.
Below is simplified sbt project source from where we attempt to resolve our sbt plugin :-
build.sbt :-
name := "ResolveBasePlugin"
build.properties :-
sbt.version=1.2.8
plugins.sbt :-
resolvers += Opts.resolver.mavenLocalFile
addSbtPlugin("io.github.moglideveloper" % "base-plugin" % "0.1")
addSbtPlugin("org.xerial.sbt" % "sbt-pack" % "0.13")
Link to simplified plugin :-
https://github.com/moglideveloper/BasePlugin
Link to simplified sbt project that attempts to resolve base-plugin
https://github.com/moglideveloper/ResolveCheck
Kindly suggest, what changes shall we do to BasePlugin, so that it can be resolved in sbt projects
that are using sbt.version=1.x
For eg :-
sbt pack plugin is published with sbt.version=1.2.8, still it can be resolved
from sbt projects with sbt.version=1.4.5
https://github.com/xerial/sbt-pack/blob/master/project/build.properties
I think you need the following in your project/plugins.sbt:
// bad: resolvers += Opts.resolver.mavenLocalFile
// good
resolvers += Resolver.mavenLocal
See Local Maven resolvers.
Note
Can you please explain what is the difference between the two?
Apache Ivy and Coursier both have resolvers for Maven-layout repositories and Ivy-layout repositories, so within sbt they need to be expressed using different types.
Resolver.mavenLocal has the static type of MavenRepository represented by MavenCache and would translate to a "plugin-capable" IBiblioResolver for Ivy. On the other hand, Opts.resolver.mavenLocalFile is a filesystem resolver:
val mavenLocalFile = Resolver.file("Local Repository", userHome / ".m2" / "repository")(
Resolver.defaultPatterns
)
This would translate to an Ivy repository with Maven-like pattern. It might work for some artifacts, but apparently not for sbt plugins. In general, I can't think of a situation Opts.resolver.mavenLocalFile must be used.
I am trying to integrate ignite in scala code and run the application using sbt. I cannot use any IDE for this.
Scala version - 2.11.0
Spark version - 2.3.0
Ignite version - 2.8.0
Sbt version - 1.3.3
I have tried adding the basic library dependency in sbt.build,
libraryDependencies += "org.apache.ignite" %% "ignite-spark" % "2.8.0"
my complete sbt.build code is,
scalaVersion := "2.11.0"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.0"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.3.0"
libraryDependencies += "org.apache.ignite" %% "ignite-spark" % "2.8.0"
but it is still not working, I am getting the following error:
$ sbt package
[info] Loading project definition from /home/testing/project
[info] Loading settings for project testing from build.sbt ...
[info] Set current project to testing (in build file:/home/testing/)
[info] Updating
[info] Resolved dependencies
[warn]
[warn] Note: Unresolved dependencies path:
[error] sbt.librarymanagement.ResolveException: Error downloading org.apache.ignite:ignite-spark_2.11:2.8.0
[error] Not found
[error] Not found
[error] not found: /root/.ivy2/local/org.apache.ignite/ignite-spark_2.11/2.8.0/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/ignite/ignite-spark_2.11/2.8.0/ignite-spark_2.11-2.8.0.pom
[error] at lmcoursier.CoursierDependencyResolution.unresolvedWarningOrThrow(CoursierDependencyResolution.scala:245)
[error] at lmcoursier.CoursierDependencyResolution.$anonfun$update$34(CoursierDependencyResolution.scala:214)
[error] at scala.util.Either$LeftProjection.map(Either.scala:573)
[error] at lmcoursier.CoursierDependencyResolution.update(CoursierDependencyResolution.scala:214)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:60)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:52)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:102)
[error] at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:69)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$20(LibraryManagement.scala:115)
[error] at scala.util.control.Exception$Catch.apply(Exception.scala:228)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:115)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:96)
[error] at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:150)
[error] at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:129)
[error] at sbt.Classpaths$.$anonfun$updateTask0$5(Defaults.scala:2946)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:281)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:19)
[error] at sbt.Execute.work(Execute.scala:290)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:281)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.apache.ignite:ignite-spark_2.11:2.8.0
[error] Not found
[error] Not found
[error] not found: /root/.ivy2/local/org.apache.ignite/ignite-spark_2.11/2.8.0/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/ignite/ignite-spark_2.11/2.8.0/ignite-spark_2.11-2.8.0.pom
[error] Total time: 5 s, completed May 28, 2020 6:24:04 AM
I am working on Ubuntu:latest docker image.
I am sure that spark and ignite are working as I am also running pyspark and it is working just fine. Please help me out, I think being a newbie I am doing some minor mistake which is becoming a major issue here.
It seems that he tries to pool "ignite_spark_2.11", not only "ignite_spark".
"not found: https://repo1.maven.org/maven2/org/apache/ignite/ignite-spark_2.11/2.8.0/ignite-spark_2.11-2.8.0.pom"
use
libraryDependencies += "org.apache.ignite" % "ignite-spark" % "2.8.0"
instead of
libraryDependencies += "org.apache.ignite" %% "ignite-spark" % "2.8.0"
%% -> get dependencies with scala version appended
% -> get dependencies without scala version appended
I have a plugin called MyPlugin which needs to update a value from another plugin. I have thus added that plugin as a dependency based on
Logging from an sbt plugin
Need to provide a SettingKey from a plugin I use in my sbt plugin
However, sbt fails to resolve the dependency fm-sbt-s3-resolver. What is the appropriate way of including the dependency on the project?
I have tried adding it to enablePlugins on MyPlugin's build.sbt as well as adding it to the dependencies like this:
libraryDependencies ++= Seq(
"com.amazonaws" % "aws-java-sdk-sts" % amazonSDKVersion,
"com.frugalmechanic" % "fm-sbt-s3-resolver" % "0.17.0"
)
In MyPlugin.scala:
import fm.sbt.S3ResolverPlugin
object MyPlugin {
override def requires = S3ResolverPlugin
override lazy val globalSettings = Seq(
resolvers += repos
S3ResolverPlugin.autoImport.s3CredentialsProvider := s3CredentialsProviderChain
)
}
I get this error from sbt:
I can find the project on this url on the Maven Central Repo.
module not found: com.frugalmechanic#fm-sbt-s3-resolver;0.17.0
[warn] ==== typesafe-ivy-releases: tried
[warn] https://repo.typesafe.com/typesafe/ivy-releases/com.frugalmechanic/fm-sbt-s3-resolver/0.17.0/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.frugalmechanic/fm-sbt-s3-resolver/0.17.0/ivys/ivy.xml
[warn] ==== local: tried
[warn] /Users/localuser/.ivy2/local/com.frugalmechanic/fm-sbt-s3-resolver/0.17.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/frugalmechanic/fm-sbt-s3-resolver/0.17.0/fm-sbt-s3-resolver-0.17.0.pom
[warn] ==== local-preloaded-ivy: tried
[warn] /Users/localuser/.sbt/preloaded/com.frugalmechanic/fm-sbt-s3-resolver/0.17.0/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:////Users/localuser/.sbt/preloaded/com/frugalmechanic/fm-sbt-s3-resolver/0.17.0/fm-sbt-s3-resolver-0.17.0.pom
[warn] ==== sonatype-public: tried
[warn] https://oss.sonatype.org/content/repositories/public/com/frugalmechanic/fm-sbt-s3-resolver/0.17.0/fm-sbt-s3-resolver-0.17.0.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.frugalmechanic#fm-sbt-s3-resolver;0.17.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] com.frugalmechanic:fm-sbt-s3-resolver:0.17.0 (/Users/localuser/lendi/core/scala/lendi-plugin/build.sbt#L13-17)
[warn] +- com.mypackage.my-plugin:0.0.6 (scalaVersion=2.12, sbtVersion=1.0)
[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.frugalmechanic#fm-sbt-s3-resolver;0.17.0: not found
[error] at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error] at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error] at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error] at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error] at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error] at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error] at xsbt.boot.Using$.withResource(Using.scala:10)
[error] at xsbt.boot.Using$.apply(Using.scala:9)
[error] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error] at xsbt.boot.Locks$.apply(Locks.scala:28)
[error] at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error] at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error] at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error] at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error] at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error] at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error] at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error] at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error] at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2561)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error] at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:278)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: com.frugalmechanic#fm-sbt-s3-resolver;0.17.0: not found
Looking into frugalmechanic group in maven,
It seems that following pom exists:
fm-sbt-s3-resolver_2.12_1.0/0.17.0/fm-sbt-s3-resolver-0.17.0.pom
^^^^^^^^^ this is scala version, and sbt version.
This path is not a normal one which consists of group id, and artifact id package version since it has _2.12_1.0 in it. This means that plugin is targeted to scala version 2.12, and sbt version 1.0.
Therefore adding plugin as dependency, you have to use Defaults.sbtPluginExtra() and tell that you need specific scala version and plugin version, as below:
libraryDependencies += Defaults.sbtPluginExtra("com.frugalmechanic" % "fm-sbt-s3-resolver" % "0.17.0", "1.0", "2.12")
first argument is package, second argument is sbt version, and third argument is scala version.
You probably want addSbtPlugin so you do not have to worry about the Scala or SBT version (it will be handled automatically):
addSbtPlugin("com.frugalmechanic" % "fm-sbt-s3-resolver" % "0.17.0")
Here is an example of it being used in another SBT plugin: built.sbt for fm-sbt-common
I am trying to run Hello_world in scala , I have installed scala and spark with sbt.
but when i check scala -version it shows
Scala code runner version 2.12.6 -- Copyright 2002-2018, LAMP/EPFL and Lightbend, Inc.
But if i am checking spark version its showing :
spark-submit --version
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.3.1
/_/
Using Scala version 2.11.8, Java HotSpot(TM) 64-Bit Server VM, 1.8.0_181
Branch
Compiled by user vanzin on 2018-06-01T20:37:04Z
Revision
Url
Type --help for more information.
Now my sbt file looks like this:
name := "HelloScala"
version := "0.1"
scalaVersion := "2.11.8"
libraryDependencies += "org.apache.spark" % "spark-core" % "2.3.1"
I also tried '2.12.6' but i am keep getting this error:
Error while importing sbt project:
[info] Loading settings for project global-plugins from idea.sbt ...
[info] Loading global plugins from /Users/monk/.sbt/1.0/plugins
[info] Loading project definition from /Users/monk/Desktop/HelloScala/project
[info] Loading settings for project helloscala from build.sbt ...
[info] Set current project to HelloScala (in build file:/Users/monk/Desktop/HelloScala/)
[info] sbt server started at local:///Users/monk/.sbt/1.0/server/cab071a62161b47022c3/sock
sbt:HelloScala>
[info] Defining Global / sbtStructureOptions, Global / sbtStructureOutputFile, shellPrompt
[info] The new values will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to HelloScala (in build file:/Users/monk/Desktop/HelloScala/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from /Users/monk/Library/Application Support/IdeaIC2018.2/Scala/launcher/sbt-structure-1.0.jar
[info] Reapplying settings...
[info] Set current project to HelloScala (in build file:/Users/monk/Desktop/HelloScala/)
[info] Updating ...
[warn] module not found: org.apache.spark#spark-core;2.3.1
[warn] ==== local: tried
[warn] /Users/monk/.ivy2/local/org.apache.spark/spark-core/2.3.1/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/org/apache/spark/spark-core/2.3.1/spark-core-2.3.1.pom
[warn] ==== local-preloaded-ivy: tried
[warn] /Users/monk/.sbt/preloaded/org.apache.spark/spark-core/2.3.1/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:////Users/monk/.sbt/preloaded/org/apache/spark/spark-core/2.3.1/spark-core-2.3.1.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.spark#spark-core;2.3.1: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.apache.spark:spark-core:2.3.1 (/Users/monk/Desktop/HelloScala/build.sbt#L4-5)
[warn] +- default:helloscala_2.11:0.1
[error] sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core;2.3.1: not found
[error] at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error] at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error] at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error] at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error] at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error] at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error] at xsbt.boot.Using$.withResource(Using.scala:10)
[error] at xsbt.boot.Using$.apply(Using.scala:9)
[error] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error] at xsbt.boot.Locks$.apply(Locks.scala:28)
[error] at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error] at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error] at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error] at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error] at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error] at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error] at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error] at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error] at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2539)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error] at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:278)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core;2.3.1: not found
[error] at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error] at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error] at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error] at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error] at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error] at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error] at xsbt.boot.Using$.withResource(Using.scala:10)
[error] at xsbt.boot.Using$.apply(Using.scala:9)
[error] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error] at xsbt.boot.Locks$.apply(Locks.scala:28)
[error] at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error] at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error] at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error] at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error] at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error] at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error] at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error] at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error] at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error] at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error] at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error] at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error] at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2539)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error] at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:278)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core;2.3.1: not found
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core;2.3.1: not found
[error] Total time: 10 s, completed 7 Sep, 2018 5:25:02 PM
[info] shutting down server
How to load or install any plugin via sbt in intelliJ ?
The problem is because you're importing your Spark dependency without %%.
Indeed, sbt requires the double percent in order to download the proper dependency for your Scala version:
If you use groupID %% artifactID % revision rather than groupID % artifactID % revision (the difference is the double %% after the groupID), sbt will add your project’s binary Scala version to the artifact name.
See here for a detailed explanation.
Wrapping up, in your case you should write:
name := "HelloScala"
version := "0.1"
scalaVersion := "2.11.8"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.1"