What is the repository for scrooge-sbt-plugin? - scala

What is the repository for the current version of scrooge-sbt-plugin? Or are the setup instructions outdated?
According to the documentation, I added this to a Play Framework project:
In project/plugins.sbt
addSbtPlugin("com.twitter" %% "scrooge-sbt-plugin" % "3.3.2")
In build.sbt:
com.twitter.scrooge.ScroogeSBT.newSettings
libraryDependencies ++= Seq(
"org.apache.thrift" % "libthrift" % "0.8.0",
"com.twitter" %% "scrooge-core" % "3.3.2",
"com.twitter" %% "finagle-thrift" % "6.5.0"
)
After play clean-all and play-compile I get this output:
[warn] module not found: com.twitter#scrooge-sbt-plugin;3.3.2
[warn] ==== typesafe-ivy-releases: tried
[warn] http://repo.typesafe.com/typesafe/ivy-releases/com.twitter/scrooge-sbt-plugin/scala_2.10/sbt_0.13/3.3.2/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn] http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.twitter/scrooge-sbt-plugin/scala_2.10/sbt_0.13/3.3.2/ivys/ivy.xml
[warn] ==== local: tried
[warn] /opt/play-2.2.0/repository/local/com.twitter/scrooge-sbt-plugin/scala_2.10/sbt_0.13/3.3.2/ivys/ivy.xml
[warn] ==== Maven2 Local: tried
[warn] file:/home/fernando/.m2/repository/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ==== sonatype-oss-snapshots: tried
[warn] http://oss.sonatype.org/content/repositories/snapshots/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ==== public: tried
[warn] http://repo1.maven.org/maven2/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ==== Typesafe repository: tried
[warn] http://repo.typesafe.com/typesafe/releases/com/twitter/scrooge-sbt-plugin_2.10_0.13/3.3.2/scrooge-sbt-plugin-3.3.2.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.twitter#scrooge-sbt-plugin;3.3.2: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Some unresolved dependencies have extra attributes. Check that these dependencies exist with the requested attributes.
[warn] com.twitter:scrooge-sbt-plugin:3.3.2 (sbtVersion=0.13, scalaVersion=2.10)
[warn]
sbt.ResolveException: unresolved dependency: com.twitter#scrooge-sbt-plugin;3.3.2: not found
There seems to be a version 3.3.1 at maven.twttr.com. What about version 3.3.2? I couldn't find it at mvnrepository.com or oss.sonatype.org.

The repository is on https://oss.sonatype.org/content/groups/public.
With a look into the Build.scala you can find out to which repository they publish.
If you look into https://oss.sonatype.org/content/groups/public/com/twitter/ and search for "scrooge-sbt-plugin" you fill find folders that ends with "_0.12", so it is published there as SBT 0.12.x plugin. You probably can't use this plugin for Play 2.2.x since it uses SBT 0.13.x.
Version 3.3.2 is not the latest release and I had problems to resolve all files. Using 3.9.2 works:
project/build.properties mus contain SBT 0.12 (example):
sbt.version=0.12.2
project/plugins.sbt must contain the resolver:
resolvers += "sonatype" at "https://oss.sonatype.org/content/groups/public"
addSbtPlugin("com.twitter" %% "scrooge-sbt-plugin" % "3.9.2")
And finally build.sbt or Build.scala must contain:
com.twitter.scrooge.ScroogeSBT.newSettings
scalaVersion := "2.10.1"
libraryDependencies ++= Seq(
"org.apache.thrift" % "libthrift" % "0.8.0",
"com.twitter" %% "scrooge-core" % "3.9.2",
"com.twitter" %% "finagle-thrift" % "6.5.0"
)

First of all, version 3.16.3 is the latest version for sbt 0.13.x
What do you mean by repository?
The code repository is on GitHub, all of the Scrooge stuff including the sbt-plugin stuff are all there.
As for the artifact repository, I am pretty sure it's on maven central or some other standard repo. You should not have to add a resolver to your sbt build. But if you do for some reason than the sonatype one mentioned by #Schleichardt seems about right.

Related

Add a maven dependency to SBT project

I am trying to reference a maven project dependency in my build.sbt file. I understand I need to add an additional resolver to my file since the project is hosted in an internal artifactory store
build.sbt
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.4.4",
"org.apache.spark" %% "spark-sql" % "2.4.4",
"com.<companyname>" %% "<libraryname>" % "2.3.0"
)
resolvers += "<library name>" at "http://artifactory.<internal url>.io:80/dsc-mvn"
However, it turns out that SBT ends up searching for a path that has _2.11 version appended to it. This is the error message that I see in IntelliJ
[info] Loading settings for project sbt-demo from build.sbt ...
[warn] Discarding 1 session setting. Use 'session save' to persist session settings.
[info] Set current project to SparkExample (in build file:<project_path>)
[info] Defining Global / sbtStructureOptions
[info] The new value will be used by Global / ssOptions
[info] Reapplying settings...
[info] Set current project to SparkExample (in build file:<project_path>)
[info] Updating ...
[warn] module not found: com.<companyname>#<libraryname>_2.11;2.3.0
[warn] ==== local: tried
[warn] /Users/vshah/.ivy2/local/com.<companyname>/<libraryname>_2.11/2.3.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
[warn] ==== local-preloaded-ivy: tried
[warn] /Users/vshah/.sbt/preloaded/com.<companyname>/<libraryname>_2.11/2.3.0/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:////Users/vshah/.sbt/preloaded/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
[warn] ==== <libraryname>: tried
[warn] http://artifactory.<internal url>.io:80/dsc-mvn/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.<companyname>#<libraryname>_2.11;2.3.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
Invalid Path (as the project is trying to search above)-
http://artifactory.<internal url>.io/dsc-mvn/com/<companyname>/<libraryname>_2.11/2.3.0/<libraryname>_2.11-2.3.0.pom
Valid path -
http://artifactory.<internal url>.io/dsc-mvn/com/<companyname>/<libraryname>/2.3.0/<libraryname>_2.11-2.3.0.pom
My question is how do I let SBT know that I am referencing a maven project at the above mentioned valid path?
This should work:
"com.<companyname>" % "<libraryname>" % "2.3.0"
The %% in the definition automatically appends the project scala version to the library name. Check out the official sbt documentation for more info.
refer to https://mvnrepository.com/artifact/org.scala-sbt to get the right Build tool for your project, and kindly check if the version in the dependency matches the version of your scala .
Can you please tell me more about the kind of dependency you need?

SBT module not found module not found: com.microsoft.sqlserver#sqljdbc4;4.0

I have a Scala app developed using IntelliJ.
When I try to package it using sbt, I get the following error and it complains about the sqljdbc4 which I have included in my build.sbt file.
> package
[info] Updating {file:/C:/Users/jthomas1/IdeaProjects/AvroToParquet/}avrotoparquet...
[info] Resolving com.microsoft.sqlserver#sqljdbc4;4.0 ...
[warn] module not found: com.microsoft.sqlserver#sqljdbc4;4.0
[warn] ==== local: tried
[warn] C:\Users\jthomas1\.ivy2\local\com.microsoft.sqlserver\sqljdbc4\4.0\ivys\ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/com/microsoft/sqlserver/sqljdbc4/4.0/sqljdbc4-4.0.pom
[warn] ==== local-preloaded-ivy: tried
[warn] C:\Users\jthomas1\.sbt\preloaded\com.microsoft.sqlserver\sqljdbc4\4.0\ivys\ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:/C:/Users/jthomas1/.sbt/preloaded/com/microsoft/sqlserver/sqljdbc4/4.0/sqljdbc4-4.0.pom
[info] Resolving jline#jline;2.11 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.microsoft.sqlserver#sqljdbc4;4.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] com.microsoft.sqlserver:sqljdbc4:4.0 (C:\Users\jthomas1\IdeaProjects\AvroToParquet\build.sbt#L13-14)
[warn] +- default:avrotoparquet_2.11:0.1
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: com.microsoft.sqlserver#sqljdbc4;4.0: not found
[error] Total time: 4 s, completed Nov 2, 2017 5:18:51 PM
Following from by build.sbt file
name := "AvroToParquet"
version := "0.1"
scalaVersion := "2.11.1"
libraryDependencies += "com.microsoft.sqlserver"
% "sqljdbc4" % "4.0"
% "runtime"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "1.6.1",
"org.apache.spark" %% "spark-sql" % "1.6.1",
"org.apache.spark" %% "spark-hive" % "1.6.1"
)
Appreciate your help!
You can add
resolvers += "Clojars" at "http://clojars.org/repo/"
to your build.sbt and it should solve the problem.
Note, that there are similar questions already and in the answers people write
Unfortunately Microsoft doesn't make this artifact available via any maven repository. You need to download the jar from the Microsoft website, and then manually install it into your local maven repository.
So if you don't want to use Clojars repository from above, you can just download this jar from Microsoft and place it in the lib/ folder in your sbt project.
Also see this answer which refers to another artifact for this driver available in Maven Central.
A Microsoft Sql Server JDBC is listed in maven central here.
There is a tab on that page listing sbt info for this jar:
libraryDependencies += "com.microsoft.sqlserver" % "mssql-jdbc" % "7.4.1.jre12"

Why is ScalaTest 3.0.1 for 2.12 not resolved?

I have a scala project that is cross built for 2.11 and 2.12
crossScalaVersions := Seq("2.11.8", "2.12.0")
I've added dependency for scalatest.
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.1" % "test",
"org.mockito" % "mockito-all" % "1.10.19" % "test"
)
resolvers ++= Seq(
"Akka" at "http://akka.io/repository/",
"Sonatype" at "http://oss.sonatype.org/content/repositories/releases"
)
I can see that 2.12 version is published for scalatest from 3.0.0. SBT is not able to resolve it though.
[warn] ==== sbt-releases-repo: tried
[warn] http://repo.typesafe.com/typesafe/ivy-releases/org.scalatest/scalatest_2.12/3.0.1/ivys/ivy.xml
[warn] ==== sbt-plugins-repo: tried
[warn] http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.scalatest/scalatest_2.12/3.0.1/ivys/ivy.xml
[warn] ==== my-ivy-proxy-releases: tried
[warn] http://artifactory.internal.com/ivy-releases/org.scalatest/scalatest_2.12/3.0.1/ivys/ivy.xml
[warn] ==== my-maven-proxy-releases: tried
[warn] http://artifactory.internal.com/maven-releases/org/scalatest/scalatest_2.12/3.0.1/scalatest_2.12-3.0.1.pom [warn] ==== Akka: tried
[warn] http://akka.io/repository/org/scalatest/scalatest_2.12/3.0.1/scalatest_2.12-3.0.1.pom
[warn] ==== Sonatype: tried
[warn] http://oss.sonatype.org/content/repositories/releases/org/scalatest/scalatest_2.12/3.0.1/scalatest_2.12-3.0.1.pom [info] Resolving jline#jline;2.14.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.scalatest#scalatest_2.12;3.0.1: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
Any pointers to what is missing here?
UPDATE:
Adding my ~/.sbt/repositories setting
[repositories]
local
my-ivy-proxy-releases: http://artifactory.internal.com/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
my-maven-proxy-releases: http://artifactory.internal.com/maven-releases/
According to your log, SBT doesn't search for the dependency on Maven Central, which is the most straightforward place to get Scalatest from. This is not an issue with a default SBT config (since Maven Central is one of the default resolvers); but here, you have a specific SBT config, to use your organization's internal repos instead (on http://artifactory.internal.com).
You need to either add Maven Central in your SBT resolvers (but that's probably not what you want, in the long term) or check with your Artifactory admins if the artifactory.internal.com/maven-releases repo is the one that's supposed to proxy Maven Central dependencies, and if it's properly configured. I think they should also be able to export from Artifactory a Maven configuration file pointing to the repos you need to use: you could then just copy and paste the URLs from that to your SBT config file.

Fail to upload to sonatype with sbt 0.13 with xsbt-gpg-plugin unresolved dependency: com.jsuereth#xsbt-gpg-plugin;0.6

my plugins.sbt has this:
logLevel := Level.Warn
addSbtPlugin("com.earldouglas" % "xsbt-web-plugin" % "1.0.0")
resolvers += Resolver.url("sbt-plugin-releases", new URL("http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases")) (Resolver.ivyStylePatterns)
addSbtPlugin("com.jsuereth" % "xsbt-gpg-plugin" % "0.6")
when i run sbt publish i get:
[warn] ==== sbt-plugin-releases: tried [warn]
http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases/com.jsuereth/xsbt-gpg-plugin/scala_2.10/sbt_0.13/0.6/ivys/ivy.xml
[warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
UNRESOLVED DEPENDENCIES :: [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
com.jsuereth#xsbt-gpg-plugin;0.6: not found [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note:
Some unresolved dependencies have extra attributes. Check that these
dependencies exist with the requested attributes. [warn]
com.jsuereth:xsbt-gpg-plugin:0.6 (sbtVersion=0.13,
scalaVersion=2.10) [warn] sbt.ResolveException: unresolved
dependency: com.jsuereth#xsbt-gpg-plugin;0.6: not found at
sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:217) at
sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:126)
So I tried updating the addSbtPlugin to be:
addSbtPlugin("com.jsuereth" % "xsbt-gpg-plugin" % "0.6",
sbtVersion = "0.12", // SBT version
scalaVersion = "2.9.2" )
Now I get
[warn] The global sbt directory is now versioned and is located at /home//.sbt/0.13.
[warn] You are seeing this warning because there is global configuration in /home//.sbt but not in /home//.sbt/0.13.
[warn] The global sbt directory may be changed via the sbt.global.base system property.
[info] Loading project definition from /home//dev/projects/myproj/project
[warn] Multiple resolvers having different access mechanism configured with same name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`).
**java.lang.NoClassDefFoundError: sbt/Scoped$ListSetting**
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
anyway to fix it please?
thanks
Not sure, but I think you use a rather old version of that plugin?
I use the following with latest sbt 0.13.6. I have this globally installed in ~/.sbt/0.13/plugins/build.sbt:
addSbtPlugin("com.typesafe.sbt" % "sbt-pgp" % "0.8.1") // sign Sonatype artifacts
I also left my credentials in ~/.sbt/0.13/sonatype.sbt:
credentials += Credentials("Sonatype Nexus Repository Manager",
"oss.sonatype.org",
"user-name", "pass-word")
This way I can do sbt publish-signed.
Here is the plugin documentation. (As you can see, there is already an even newer version available)

SBT not resolving Squeryl dependency

I recently started a new project with the Play! Framework and Scala.
I'm used to using Squeryl for my ORM, but for some reason it cannot resolve my dependency this time (Although it will resolve others, just not squeryl).
The only thing I'm doing differently is that I'm on a different computer than I was before (Windows now, Arch before) and I'm using Play 2.1.1 instead of 2.1.
EDIT: I am also behind a proxy, I thought this may have been resolved since I can resolve some dependencies, but I can't see any other reason than the proxy is screwing with sbt. I can see the maven repo for squeryl in my browser, but sbt fails to find it.
build.properties:
sbt.version=0.12.2
Build.scala:
val appDependencies = Seq(
// Add your project dependencies here,
jdbc,
"org.squeryl" %% "squeryl" % "0.9.5-6"
)
plugins.sbt:
// Comment to get more information during initialization
logLevel := Level.Warn
// The Typesafe repository
resolvers += "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"
// Use the Play sbt plugin for Play projects
addSbtPlugin("play" % "sbt-plugin" % "2.1.1")
Console:
C:\Path\To\Play\APP>play run
[info] Loading project definition from C:\Path\To\Play\APP
....
[warn] module not found: org.squeryl#squeryl_2.10;0.9.5-6
[warn] ==== local: tried
[warn] C:\Path\To\Play\play-2.1.1\repository\local\org.squeryl\squeryl_2.10
[warn] ==== Typesafe Releases Repository: tried
[warn] http://repo.typesafe.com/typesafe/releases/org/squeryl/squeryl_2.10/0.9.5-6/squeryl_2.10-0.9.5-6.po
[warn] ==== Typesafe Snapshots Repository: tried
[warn] http://repo.typesafe.com/typesafe/snapshots/org/squeryl/squeryl_2.10/0.9.5-6/squeryl_2.10-0.9.5-6.p
[warn] ==== public: tried
[warn] http://repo1.maven.org/maven2/org/squeryl/squeryl_2.10/0.9.5-6/squeryl_2.10-0.9.5-6.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.squeryl#squeryl_2.10;0.9.5-6: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency: org.squeryl#squeryl_2.10;0.9.5-6: not found
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:214)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122)
...
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
[error] (*:update) sbt.ResolveException: unresolved dependency: org.squeryl#squeryl_2.10;0.9.5-6: not found
[warn] some of the dependencies were not recompiled properly, so classloader is not avaialable
[info] Updating {file:/C:/Path/To/Play/APP}
[warn] module not found: org.squeryl#squeryl_2.10;0.9.5-6
[warn] ==== local: tried
[warn] C:\Path\To\Play\play-2.1.1\repository\local\org.squeryl\squeryl_2.10
[warn] ==== Typesafe Releases Repository: tried
[warn] http://repo.typesafe.com/typesafe/releases/org/squeryl/squeryl_2.10/0.9.5-6/squeryl_2.10-0.9.5-6.po
[warn] ==== Typesafe Snapshots Repository: tried
[warn] http://repo.typesafe.com/typesafe/snapshots/org/squeryl/squeryl_2.10/0.9.5-6/squeryl_2.10-0.9.5-6.p
[warn] ==== public: tried
[warn] http://repo1.maven.org/maven2/org/squeryl/squeryl_2.10/0.9.5-6/squeryl_2.10-0.9.5-6.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.squeryl#squeryl_2.10;0.9.5-6: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency: org.squeryl#squeryl_2.10;0.9.5-6: not found
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:214)
.....
at java.lang.Thread.run(Unknown Source)
[error] (*:update) sbt.ResolveException: unresolved dependency: org.squeryl#squeryl_2.10;0.9.5-6: not found
When you see http://repo1.maven.org/maven2/org/squeryl/
you will get solution.
PROJECT/project/Build.scala
val appDependencies = Seq(
// Add your project dependencies here,
jdbc,
anorm,
"org.squeryl" % "squeryl_2.10" % "0.9.5-6"
)
This is from my build.sbt (well, a relevant section) - what scala version?
scalaVersion := "2.10.1"
resolvers += "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases"
libraryDependencies ++= Seq(
"org.squeryl" %% "squeryl" % "0.9.5-6",
To be sure that it's not a potential SBT project's configuration issue, don't use for now the %% notation. Indeed, this one automatically chooses the Jar version corresponding to your current scala version, which may be different than the one you expect (oversight in your conf, conflict of variables in some configuration files etc...).
Prefer this in order to isolate your "error" context:
libraryDependencies += "org.squeryl" % "squeryl" % "0.9.5-6"
It ended up being an issue with my proxy at work, it was setup wrong and had to correct it. All is well now!