How to toggle between project and library dependencies in SBT? - scala

It's easy to declare managed library dependencies in SBT, eg
libraryDependencies ++= Seq(
"org.specs2" %% "specs2" % "1.12.2" % "test" ,
"junit" % "junit" % "4.7" % "test"
)
And while it's not as easy to declare project dependencies in SBT, I can do that too:
object RichMath extends Build {
lazy val myApp = Project("RichMath", file(".")) dependsOn(richUtil)
lazy val richUtil = RootProject(file("../RichUtil"))
}
But in practice, I typically want to change between project mode, where changes are immediately visible in upstream projects, and library mode, where I must publish changes to see them in dependent projects, as code matures.
Early in code-base's life, or whenever I'm wanting to make frequent changes across modules, I don't want the hassle of re-publishing just to see changes upstream. But in stable/mature code, I want to specify exactly what version's I'm depending upon.
It seems like SBT treats the two dependencies as completely different. Is there a more straight-forward way to switch between project- and library- dependencies than rewriting my build definition?

I have a few scenarios for my sbt scripts (tests, publishing, production). I start sbt from script (from bash, you may have other environment) with DO=TESTS sbt for example. This is my dynamic dependencies with regard of environment variable:
if (sys.env.contains("LOCAL_BUILD")) {
Seq[Project.Setting[_]](
unmanagedResourceDirectories in Compile <+= baseDirectory { _ / "src" / "main" / "scala" },
libraryDependencies ++= {
Seq(
"org.digimead" %% "digi-lib-slf4j" % "0.2.1-SNAPSHOT" % "test",
"org.digimead" %% "digi-lib-test" % "0.2.1-SNAPSHOT" % "test",
"org.scalatest" %% "scalatest" % "1.9" % "test"
)
}
)
} else {
Seq[Project.Setting[_]](
libraryDependencies ++= {
Seq(
"org.slf4j" % "slf4j-log4j12" % "1.7.1"
)
}
)
}
As you can see I may have different project settings with single .sbt definition controlled by one environment variable. The environment variable affect all project/subproject bunch.

It is true that the two types of dependencies are treated rather differently and it would be nice if they were not. The main obstacle is that sbt needs to know about all external projects before settings are loaded (for various reasons).
For now, the easiest solution is probably an environment variable or system property as described in another answer. Going forward, the following is very close to being possible in sbt, but still needs some more work:
Declare a dependency as usual
libraryDependencies += "org.example" % "rich-util" % "0.1"
Add the source dependency from the command line, overriding the normal dependency automatically in the process
$ sbt
> projects add ../RichUtil
The convention-based approach described in Setting up sbt environment to hack on multiple libraries at once is a special case and would be enabled by this working as well.

Related

dependency resolution with sbt for continuous integration

I'm an open-source Scala developer who wants to minimize the hassle of dependencies when pushing to GitHub and triggering a check with continuous integration (CircleCI). I have two projects where one (A) is dependent on the other (B). B is under development at the same time (as a snapshot). My project A build.sbt file has a dependency on this (snapshot) version of B and of course all works fine on my local machine. When I push to GitHub, it naturally fails as that snapshot file is not available to CircleCI.
I have generally worked around this by putting the jar file into my lib directory (and removing the dependency from build.sbt). I believe this is known as an unmanaged dependency.
My question is this: is there any way of setting up my lib directory so that CircleCI can resolve the (managed) dependency from the lib directory? I have tried putting the ivy structure into lib starting with the top level com.phasmidsoftware, with b_2.13 under that and under that 1.0.4-SNAPSHOT and so on down. That doesn't work. I've attached the build.sbt for project A (called Numeric).
organization := "com.phasmidsoftware"
name := "Number"
version := "1.0.9"
scalaVersion := "2.13.6"
scalacOptions ++= Seq( "-target:jvm-1.8", "-encoding", "UTF-8", "-unchecked", "-deprecation", "-Ywarn-dead-code", "-Ywarn-value-discard", "-Ywarn-unused" )
val scalaTestVersion = "3.2.3"
libraryDependencies += "org.scalatest" %% "scalatest" % scalaTestVersion % "test"
resolvers += "Typesafe Repository" at "https://repo.typesafe.com/typesafe/releases/"
libraryDependencies ++= Seq(
"com.phasmidsoftware" %% "matchers" % "1.0.4-SNAPSHOT",
"org.scala-lang.modules" %% "scala-parser-combinators" % "1.2.0-M1",
"org.apache.commons" % "commons-math3" % "3.6.1",
"org.slf4j" % "slf4j-api" % "1.7.31",
"ch.qos.logback" % "logback-classic" % "1.2.3" % "test",
"org.scalacheck" %% "scalacheck" % "1.14.1" % "test"
)
The answer described in How can sbt pull dependency artifacts from git? is indeed the right answer.
I will just add a couple of caveats.
Make sure that you use the git protocol in the URI of your git repository (as is shown in the other answer);
The mechanism works by cloning the repository (the branch is defined by ...#branch) but, once you've cloned it, sbt won't fetch if appropriate -- you do have to do that explicitly yourself.
The other thing to keep in mind is that the clone(s) are placed in ~/.sbt/1.0/staging/... where 1.0 is based on the sbt version number.
And, of course, don't forget to remove the reference to the other project if you have it in your libraryDependencies.
Here's the relevant part of my build.sbt file:
lazy val root = (project in file(".")).dependsOn(matchers)
lazy val matchers = RootProject(uri("git://github.com/rchillyard/Matchers#V1_0_5"))

Scala library available in both compile and test configuration

I have a library that I wish to expose in both the unit tests in Scala and the code itself.
In sbt, I added my library dependency with configuration "test" and then it's available for tests but I cannot use it in the code. If I leave the configuration be or add "compile" it's not available to be imported in unit tests.
libraryDependencies ++= Seq(
"org.scalacheck" %% "scalacheck" % "1.14.0",
"org.scalatest" %% "scalatest" % "3.0.6" % "test",
"org.scalactic" %% "scalactic" % "3.0.6" % "test")
The main problem is that I expose an abstract class I want to use all over the place in other code: abstract class UnitSpec extends FlatSpec with Matchers with ScalaCHeckDrivenPropertyChecks and also use in the tests of the library. If I add "test" to ScalaCheck it cannot find it in the main code of the library. If I leave it as is, it cannot from org.scalatestplus.scalacheck.ScalaCheckDrivenPropertyChecks. This used to be OK and work fine with 3.0.5 and GeneratorDrivenProperyChecks but that's been deprecated.
Is there a way to achieve what I want? I tried "test->compile" but that also doesn't do what I had hoped...
You can combine configurations. In order to have a library both in compile and test you just add bot configurations.
// wrong: libraryDependencies += "<organization>" %% "<module>" % "<version>" % "compile->compile" % "test->compile"
The syntax means roughly: project configuration dependsOn(->) configuration of libraryDependency.
Update
You can also add the dependency twice with different configurations.
libraryDependencies += "<organization>" %% "<module>" % "<version>",
libraryDependencies += "<organization>" %% "<module>" % "<version>" % "test"
Update 2
I think the syntax in the first example is not what I meant to provide.
libraryDependencies += "<organization>" %% "<module>" % "<version>" % "compile->compile;test->compile"
At least that is what I use in my libraryDependencies.
So you need a trait from the Scalatest JAR in non-test code. I am not sure why it worked before, but it would make sense to me just to remove % "test" from the scalatest dependency. That will make it available in compile and everything from compile is available in test too.
And for Scalactic I think the main use-case for it as a separate dependency is when you need it in compile but only use Scalatest in test (or don't use it at all). If they are both needed for tests only (or for compile), Scalatest will bring Scalactic with it.
I tried "test->compile" but that also doesn't do what I had hoped...
"test->compile" is the same as "test":
A configuration without a mapping (no "->") is mapped to "default" or "compile". The -> is only needed when mapping to a different configuration than those.

SBT resolve test artifacts from Nexus. Workaround not very helpful

I'm working on a project where I need to add as a dependency one of our projects lying in our nexus repositories. I need to use the test and main classes from this project.
In my build.sbt, the following fails to resolve the dependency with a message that the test configuration is not public
libraryDependencies += "XProj" % "XProj_2.11" % "1.0-SNAPSHOT" % "test->test;test->compile" classifier "tests"
This works fine
libraryDependencies += "XProj" % "XProj_2.11" % "1.0-SNAPSHOT" % "test" classifier "tests"
But I also need to use classes from the main source folder which are not resolved this way. I tried "test;test->compile" and although the dependency is resolved, the main classes are not fetched. Calling a method from one of XProj's test classes, fails with a java.lang.NoClassDefFoundError Is there any other way to add the main classes and resolve the dependency at the same time?
Try this:
libraryDependencies ++= {
val xproj = "XProj" %% "XProj" % "1.0-SNAPSHOT"
Seq(
xproj % "test" classifier "tests"
xproj % "test->compile" classifier "tests"
)
}
Cheers
Without knowing a bit more about the configurations of your dependency, I think you probably want to change:
libraryDependencies += "XProj" % "XProj_2.11" % "1.0-SNAPSHOT" % "test" classifier "tests"
to something like:
libraryDependencies += "XProj" % "XProj_2.11" % "1.0-SNAPSHOT" % "default,test" classifier "tests"
where the default,test is a comma separated list of the configurations that you need to pull in.
The -> indicators in Ivy are generally used to indicate transitive configurations for your module, i.e. if I write test->compile this means that the test configuration of my module should pull in the compile configuration of the dependency.

Cannot Resolve Symbol "Scalatest"

I am trying to use scalatest, but Intellij cannot recognize:
import org.scalatest._
Here is my build.sbt file, located in the same directory as my scalatest.jar file.
scalaVersion := "2.11.2"
libraryDependencies += "org.scalatest" % "scalatest_2.11" % "2.2.4" % "test"
Thanks
So you have by convention two source folders:
src/main/scala/...
src/test/scala/...
The first is shown blue, the second green in IntelliJ IDEA. The library dependencies in sbt are associated with either of these, so
"org.foo" % "bar_2.11" % "1.2.3"
Is a main dependency, available to main sources (and also test, because test depends on main). And
"org.foo" % "bar_2.11" % "1.2.3" % "test"
Is a test dependency, only available to test sources. The idea is that these are libraries which are not required for your product, but only to run the unit tests.
In your example, Scala-Test is only available to test sources, so trying to import it from main sources will fail.

sbt multi-module project global version setting

I am new to SBT and trying to set up a multi-module project. I come across to a situation where I would like to have a single place where I could have defined versions for libs used accross modules. I tried following with creating a custom settingKey - in the root project:
val akkaVersion = SettingKey[String]("Akka version used in our project")
name := "hello-app"
version in ThisBuild := "1.0.0"
organization in ThisBuild := "com.jaksky.hello"
scalaVersion := "2.10.4"
akkaVersion in ThisBuild:= "2.3.4"
// Common settings/definitions for the build
def OurProject(name: String): Project = (
Project(name, file(name))
)
lazy val common = (
OurProject("common")
)
lazy val be_services = (
OurProject("be-services")
dependsOn(common)
)
In project be-services I tried following:
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion.value,
"com.typesafe.akka" %% "akka-cluster" % akkaVersion.value,
"com.typesafe.akka" %% "akka-kernel" % akkaVersion.value,
"com.typesafe.akka" %% "akka-remote" % akkaVersion.value,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion.value,
"ch.qos.logback" % "logback-classic" % "1.0.13"
)
The point here is akkaVersion is not visible (akkaVersion is not found - that is the error message).
My questins:
Is settings map shared accross modules? Described issue probably answers -> NO
What is best practice in this situation?
I found following possibilities:
Scala object holding string constants. Seems to me a bit akward as project version is specified in build.sbt so why dependant libs should be hidden somewhere in project/GlobalVersions.scala or so.
Crating a libDepenDency seq which can be reused. That limits flexibility and I do not always want to be dependant on those libs mentioned.
Custom setting seems to be a bit heavy weighed but seems to me as a claean way but wasn't able to make it work.
Just to complete the picture - using SBT 0.13.5
I believe that this is so fundamental problem that I am not the first one facing this issue.
Thanks for helping me out
The definitions in .sbt files are not visible in other .sbt files. In order to share code between .sbt files, define one or more Scala files in the project/ directory of the build root.
[from documentation http://www.scala-sbt.org/]