I am trying to reference a maven project dependency in my build.sbt file. I understand I need to add an additional resolver to my file since the project is hosted in an internal artifactory store
build.sbt
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.4.4",
"org.apache.spark" %% "spark-sql" % "2.4.4",
"com.<companyname>" %% "<libraryname>" % "2.3.0"
)
resolvers += "<library name>" at "http://artifactory.<internal url>.io:80/dsc-mvn"
However, it turns out that SBT ends up searching for a path that has _2.11 version appended to it. This is the error message that I see in IntelliJ
[info] Loading settings for project sbt-demo from build.sbt ...
[warn] Discarding 1 session setting. Use 'session save' to persist session settings.
[info] Set current project to SparkExample (in build file:<project_path>)
[info] Defining Global / sbtStructureOptions
[info] The new value will be used by Global / ssOptions
[info] Reapplying settings...
[info] Set current project to SparkExample (in build file:<project_path>)
[info] Updating ...
[warn] module not found: com.<companyname>#<libraryname>_2.11;2.3.0
[warn] ==== local: tried
[warn] /Users/vshah/.ivy2/local/com.<companyname>/<libraryname>_2.11/2.3.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
[warn] ==== local-preloaded-ivy: tried
[warn] /Users/vshah/.sbt/preloaded/com.<companyname>/<libraryname>_2.11/2.3.0/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:////Users/vshah/.sbt/preloaded/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
[warn] ==== <libraryname>: tried
[warn] http://artifactory.<internal url>.io:80/dsc-mvn/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.<companyname>#<libraryname>_2.11;2.3.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
Invalid Path (as the project is trying to search above)-
http://artifactory.<internal url>.io/dsc-mvn/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
Valid path -
http://artifactory.<internal url>.io/dsc-mvn/com/<companyname>/<libraryname>/2.3.0/<libraryname>_2.11-2.3.0.pom
My question is how do I let SBT know that I am referencing a maven project at the above mentioned valid path?
This should work:
"com.<companyname>" % "<libraryname>" % "2.3.0"
The %% in the definition automatically appends the project scala version to the library name. Check out the official sbt documentation for more info.
refer to https://mvnrepository.com/artifact/org.scala-sbt to get the right Build tool for your project, and kindly check if the version in the dependency matches the version of your scala .
Can you please tell me more about the kind of dependency you need?
Related
I have a Scala app developed using IntelliJ.
When I try to package it using sbt, I get the following error and it complains about the sqljdbc4 which I have included in my build.sbt file.
> package
[info] Updating {file:/C:/Users/jthomas1/IdeaProjects/AvroToParquet/}avrotoparquet...
[info] Resolving com.microsoft.sqlserver#sqljdbc4;4.0 ...
[warn] module not found: com.microsoft.sqlserver#sqljdbc4;4.0
[warn] ==== local: tried
[warn] C:\Users\jthomas1\.ivy2\local\com.microsoft.sqlserver\sqljdbc4\4.0\ivys\ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/microsoft/sqlserver/sqljdbc4/4.0/sqljdbc4-4.0.pom
[warn] ==== local-preloaded-ivy: tried
[warn] C:\Users\jthomas1\.sbt\preloaded\com.microsoft.sqlserver\sqljdbc4\4.0\ivys\ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:/C:/Users/jthomas1/.sbt/preloaded/com/microsoft/sqlserver/sqljdbc4/4.0/sqljdbc4-4.0.pom
[info] Resolving jline#jline;2.11 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.microsoft.sqlserver#sqljdbc4;4.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] com.microsoft.sqlserver:sqljdbc4:4.0 (C:\Users\jthomas1\IdeaProjects\AvroToParquet\build.sbt#L13-14)
[warn] +- default:avrotoparquet_2.11:0.1
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: com.microsoft.sqlserver#sqljdbc4;4.0: not found
[error] Total time: 4 s, completed Nov 2, 2017 5:18:51 PM
Following from by build.sbt file
name := "AvroToParquet"
version := "0.1"
scalaVersion := "2.11.1"
libraryDependencies += "com.microsoft.sqlserver"
% "sqljdbc4" % "4.0"
% "runtime"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "1.6.1",
"org.apache.spark" %% "spark-sql" % "1.6.1",
"org.apache.spark" %% "spark-hive" % "1.6.1"
)
Appreciate your help!
You can add
resolvers += "Clojars" at "http://clojars.org/repo/"
to your build.sbt and it should solve the problem.
Note, that there are similar questions already and in the answers people write
Unfortunately Microsoft doesn't make this artifact available via any maven repository. You need to download the jar from the Microsoft website, and then manually install it into your local maven repository.
So if you don't want to use Clojars repository from above, you can just download this jar from Microsoft and place it in the lib/ folder in your sbt project.
Also see this answer which refers to another artifact for this driver available in Maven Central.
A Microsoft Sql Server JDBC is listed in maven central here.
There is a tab on that page listing sbt info for this jar:
libraryDependencies += "com.microsoft.sqlserver" % "mssql-jdbc" % "7.4.1.jre12"
I have a scala project that is cross built for 2.11 and 2.12
crossScalaVersions := Seq("2.11.8", "2.12.0")
I've added dependency for scalatest.
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.1" % "test",
"org.mockito" % "mockito-all" % "1.10.19" % "test"
)
resolvers ++= Seq(
"Akka" at "http://akka.io/repository/",
"Sonatype" at "http://oss.sonatype.org/content/repositories/releases"
)
I can see that 2.12 version is published for scalatest from 3.0.0. SBT is not able to resolve it though.
[warn] ==== sbt-releases-repo: tried
[warn] http://repo.typesafe.com/typesafe/ivy-releases/org.scalatest/scalatest_2.12/3.0.1/ivys/ivy.xml
[warn] ==== sbt-plugins-repo: tried
[warn] http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.scalatest/scalatest_2.12/3.0.1/ivys/ivy.xml
[warn] ==== my-ivy-proxy-releases: tried
[warn] http://artifactory.internal.com/ivy-releases/org.scalatest/scalatest_2.12/3.0.1/ivys/ivy.xml
[warn] ==== my-maven-proxy-releases: tried
[warn] http://artifactory.internal.com/maven-releases/org/scalatest/scalatest_2.12/3.0.1/scalatest_2.12-3.0.1.pom [warn] ==== Akka: tried
[warn] http://akka.io/repository/org/scalatest/scalatest_2.12/3.0.1/scalatest_2.12-3.0.1.pom
[warn] ==== Sonatype: tried
[warn] http://oss.sonatype.org/content/repositories/releases/org/scalatest/scalatest_2.12/3.0.1/scalatest_2.12-3.0.1.pom [info] Resolving jline#jline;2.14.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.scalatest#scalatest_2.12;3.0.1: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
Any pointers to what is missing here?
UPDATE:
Adding my ~/.sbt/repositories setting
[repositories]
local
my-ivy-proxy-releases: http://artifactory.internal.com/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
my-maven-proxy-releases: http://artifactory.internal.com/maven-releases/
According to your log, SBT doesn't search for the dependency on Maven Central, which is the most straightforward place to get Scalatest from. This is not an issue with a default SBT config (since Maven Central is one of the default resolvers); but here, you have a specific SBT config, to use your organization's internal repos instead (on http://artifactory.internal.com).
You need to either add Maven Central in your SBT resolvers (but that's probably not what you want, in the long term) or check with your Artifactory admins if the artifactory.internal.com/maven-releases repo is the one that's supposed to proxy Maven Central dependencies, and if it's properly configured. I think they should also be able to export from Artifactory a Maven configuration file pointing to the repos you need to use: you could then just copy and paste the URLs from that to your SBT config file.
my plugins.sbt has this:
logLevel := Level.Warn
addSbtPlugin("com.earldouglas" % "xsbt-web-plugin" % "1.0.0")
resolvers += Resolver.url("sbt-plugin-releases", new URL("http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases")) (Resolver.ivyStylePatterns)
addSbtPlugin("com.jsuereth" % "xsbt-gpg-plugin" % "0.6")
when i run sbt publish i get:
[warn] ==== sbt-plugin-releases: tried [warn]
http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases/com.jsuereth/xsbt-gpg-plugin/scala_2.10/sbt_0.13/0.6/ivys/ivy.xml
[warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
UNRESOLVED DEPENDENCIES :: [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
com.jsuereth#xsbt-gpg-plugin;0.6: not found [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note:
Some unresolved dependencies have extra attributes. Check that these
dependencies exist with the requested attributes. [warn]
com.jsuereth:xsbt-gpg-plugin:0.6 (sbtVersion=0.13,
scalaVersion=2.10) [warn] sbt.ResolveException: unresolved
dependency: com.jsuereth#xsbt-gpg-plugin;0.6: not found at
sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:217) at
sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:126)
So I tried updating the addSbtPlugin to be:
addSbtPlugin("com.jsuereth" % "xsbt-gpg-plugin" % "0.6",
sbtVersion = "0.12", // SBT version
scalaVersion = "2.9.2" )
Now I get
[warn] The global sbt directory is now versioned and is located at /home//.sbt/0.13.
[warn] You are seeing this warning because there is global configuration in /home//.sbt but not in /home//.sbt/0.13.
[warn] The global sbt directory may be changed via the sbt.global.base system property.
[info] Loading project definition from /home//dev/projects/myproj/project
[warn] Multiple resolvers having different access mechanism configured with same name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`).
**java.lang.NoClassDefFoundError: sbt/Scoped$ListSetting**
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
anyway to fix it please?
thanks
Not sure, but I think you use a rather old version of that plugin?
I use the following with latest sbt 0.13.6. I have this globally installed in ~/.sbt/0.13/plugins/build.sbt:
addSbtPlugin("com.typesafe.sbt" % "sbt-pgp" % "0.8.1") // sign Sonatype artifacts
I also left my credentials in ~/.sbt/0.13/sonatype.sbt:
credentials += Credentials("Sonatype Nexus Repository Manager",
"oss.sonatype.org",
"user-name", "pass-word")
This way I can do sbt publish-signed.
Here is the plugin documentation. (As you can see, there is already an even newer version available)
What is the repository for the current version of scrooge-sbt-plugin? Or are the setup instructions outdated?
According to the documentation, I added this to a Play Framework project:
In project/plugins.sbt
addSbtPlugin("com.twitter" %% "scrooge-sbt-plugin" % "3.3.2")
In build.sbt:
com.twitter.scrooge.ScroogeSBT.newSettings
libraryDependencies ++= Seq(
"org.apache.thrift" % "libthrift" % "0.8.0",
"com.twitter" %% "scrooge-core" % "3.3.2",
"com.twitter" %% "finagle-thrift" % "6.5.0"
)
After play clean-all and play-compile I get this output:
[warn] module not found: com.twitter#scrooge-sbt-plugin;3.3.2
[warn] ==== typesafe-ivy-releases: tried
[warn] http://repo.typesafe.com/typesafe/ivy-releases/com.twitter/scrooge-sbt-plugin/scala_2.10/sbt_0.13/3.3.2/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn] http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.twitter/scrooge-sbt-plugin/scala_2.10/sbt_0.13/3.3.2/ivys/ivy.xml
[warn] ==== local: tried
[warn] /opt/play-2.2.0/repository/local/com.twitter/scrooge-sbt-plugin/scala_2.10/sbt_0.13/3.3.2/ivys/ivy.xml
[warn] ==== Maven2 Local: tried
[warn] file:/home/fernando/.m2/repository/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ==== sonatype-oss-snapshots: tried
[warn] http://oss.sonatype.org/content/repositories/snapshots/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ==== public: tried
[warn] http://repo1.maven.org/maven2/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ==== Typesafe repository: tried
[warn] http://repo.typesafe.com/typesafe/releases/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.twitter#scrooge-sbt-plugin;3.3.2: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] com.twitter:scrooge-sbt-plugin:3.3.2 (sbtVersion=0.13, scalaVersion=2.10)
[warn]
sbt.ResolveException: unresolved dependency: com.twitter#scrooge-sbt-plugin;3.3.2: not found
There seems to be a version 3.3.1 at maven.twttr.com. What about version 3.3.2? I couldn't find it at mvnrepository.com or oss.sonatype.org.
The repository is on https://oss.sonatype.org/content/groups/public.
With a look into the Build.scala you can find out to which repository they publish.
If you look into https://oss.sonatype.org/content/groups/public/com/twitter/ and search for "scrooge-sbt-plugin" you fill find folders that ends with "_0.12", so it is published there as SBT 0.12.x plugin. You probably can't use this plugin for Play 2.2.x since it uses SBT 0.13.x.
Version 3.3.2 is not the latest release and I had problems to resolve all files. Using 3.9.2 works:
project/build.properties mus contain SBT 0.12 (example):
sbt.version=0.12.2
project/plugins.sbt must contain the resolver:
resolvers += "sonatype" at "https://oss.sonatype.org/content/groups/public"
addSbtPlugin("com.twitter" %% "scrooge-sbt-plugin" % "3.9.2")
And finally build.sbt or Build.scala must contain:
com.twitter.scrooge.ScroogeSBT.newSettings
scalaVersion := "2.10.1"
libraryDependencies ++= Seq(
"org.apache.thrift" % "libthrift" % "0.8.0",
"com.twitter" %% "scrooge-core" % "3.9.2",
"com.twitter" %% "finagle-thrift" % "6.5.0"
)
First of all, version 3.16.3 is the latest version for sbt 0.13.x
What do you mean by repository?
The code repository is on GitHub, all of the Scrooge stuff including the sbt-plugin stuff are all there.
As for the artifact repository, I am pretty sure it's on maven central or some other standard repo. You should not have to add a resolver to your sbt build. But if you do for some reason than the sonatype one mentioned by #Schleichardt seems about right.
scala is version 2.10.2 and sbt is 0.13.0
I am trying to setup scct for scala unit test code coverage so in build.sbt
I added
seq(ScctPlugin.instrumentSettings : _*)
and in project/plugins.sbt I added
resolvers += Classpaths.typesafeResolver
resolvers += "scct-github-repository" at "http://mtkopone.github.com/scct/maven-repo"
addSbtPlugin("reaktor" %% "sbt-scct" % "0.2-SNAPSHOT")
I get errors like ....
module not found: reaktor#sbt-scct;0.2-SNAPSHOT
Last message is
[warn] ==== scct-github-repository: tried
[warn] http://mtkopone.github.com/scct/maven-repo/reaktor/sbt-scct_2.10_0.13/0.2-SNAPSHOT/sbt-scct-0.2-SNAPSHOT.pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: reaktor#sbt-scct;0.2-SNAPSHOT: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] reaktor:sbt-scct:0.2-SNAPSHOT (sbtVersion=0.13, scalaVersion=2.10)
I look in http://mtkopone.github.io/scct/maven-repo/reaktor/
and I see the one I seem to need is missing e.g. I see
sbt-scct_2.9.2_0.13/
scct_2.10/
and a bunch of others but no ... sbt-scct_2.10_0.13
Can you help ? (I am a sbt newbie)
SCCT has a new source code repository: https://github.com/SCCT/scct
As a result you can fetch a release version from maven central with addSbtPlugin("com.github.scct" %% "sbt-scct" % "0.2") in your plugins.sbt file.
My project to reproduce is on GitHub.
Update:
The repository has been relocated at https://github.com/sqality/scct
and you need addSbtPlugin("com.sqality.scct" % "sbt-scct" % "version") in your plugins.sbt file.
The original SCCT is inactive now.
Here are the two most active forks.
http://github.com/sqality
http://github.com/scoverage