sbt multi-module project global version setting - scala

I am new to SBT and trying to set up a multi-module project. I come across to a situation where I would like to have a single place where I could have defined versions for libs used accross modules. I tried following with creating a custom settingKey - in the root project:
val akkaVersion = SettingKey[String]("Akka version used in our project")
name := "hello-app"
version in ThisBuild := "1.0.0"
organization in ThisBuild := "com.jaksky.hello"
scalaVersion := "2.10.4"
akkaVersion in ThisBuild:= "2.3.4"
// Common settings/definitions for the build
def OurProject(name: String): Project = (
Project(name, file(name))
)
lazy val common = (
OurProject("common")
)
lazy val be_services = (
OurProject("be-services")
dependsOn(common)
)
In project be-services I tried following:
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion.value,
"com.typesafe.akka" %% "akka-cluster" % akkaVersion.value,
"com.typesafe.akka" %% "akka-kernel" % akkaVersion.value,
"com.typesafe.akka" %% "akka-remote" % akkaVersion.value,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion.value,
"ch.qos.logback" % "logback-classic" % "1.0.13"
)
The point here is akkaVersion is not visible (akkaVersion is not found - that is the error message).
My questins:
Is settings map shared accross modules? Described issue probably answers -> NO
What is best practice in this situation?
I found following possibilities:
Scala object holding string constants. Seems to me a bit akward as project version is specified in build.sbt so why dependant libs should be hidden somewhere in project/GlobalVersions.scala or so.
Crating a libDepenDency seq which can be reused. That limits flexibility and I do not always want to be dependant on those libs mentioned.
Custom setting seems to be a bit heavy weighed but seems to me as a claean way but wasn't able to make it work.
Just to complete the picture - using SBT 0.13.5
I believe that this is so fundamental problem that I am not the first one facing this issue.
Thanks for helping me out

The definitions in .sbt files are not visible in other .sbt files. In order to share code between .sbt files, define one or more Scala files in the project/ directory of the build root.
[from documentation http://www.scala-sbt.org/]

Related

dependency resolution with sbt for continuous integration

I'm an open-source Scala developer who wants to minimize the hassle of dependencies when pushing to GitHub and triggering a check with continuous integration (CircleCI). I have two projects where one (A) is dependent on the other (B). B is under development at the same time (as a snapshot). My project A build.sbt file has a dependency on this (snapshot) version of B and of course all works fine on my local machine. When I push to GitHub, it naturally fails as that snapshot file is not available to CircleCI.
I have generally worked around this by putting the jar file into my lib directory (and removing the dependency from build.sbt). I believe this is known as an unmanaged dependency.
My question is this: is there any way of setting up my lib directory so that CircleCI can resolve the (managed) dependency from the lib directory? I have tried putting the ivy structure into lib starting with the top level com.phasmidsoftware, with b_2.13 under that and under that 1.0.4-SNAPSHOT and so on down. That doesn't work. I've attached the build.sbt for project A (called Numeric).
organization := "com.phasmidsoftware"
name := "Number"
version := "1.0.9"
scalaVersion := "2.13.6"
scalacOptions ++= Seq( "-target:jvm-1.8", "-encoding", "UTF-8", "-unchecked", "-deprecation", "-Ywarn-dead-code", "-Ywarn-value-discard", "-Ywarn-unused" )
val scalaTestVersion = "3.2.3"
libraryDependencies += "org.scalatest" %% "scalatest" % scalaTestVersion % "test"
resolvers += "Typesafe Repository" at "https://repo.typesafe.com/typesafe/releases/"
libraryDependencies ++= Seq(
"com.phasmidsoftware" %% "matchers" % "1.0.4-SNAPSHOT",
"org.scala-lang.modules" %% "scala-parser-combinators" % "1.2.0-M1",
"org.apache.commons" % "commons-math3" % "3.6.1",
"org.slf4j" % "slf4j-api" % "1.7.31",
"ch.qos.logback" % "logback-classic" % "1.2.3" % "test",
"org.scalacheck" %% "scalacheck" % "1.14.1" % "test"
)
The answer described in How can sbt pull dependency artifacts from git? is indeed the right answer.
I will just add a couple of caveats.
Make sure that you use the git protocol in the URI of your git repository (as is shown in the other answer);
The mechanism works by cloning the repository (the branch is defined by ...#branch) but, once you've cloned it, sbt won't fetch if appropriate -- you do have to do that explicitly yourself.
The other thing to keep in mind is that the clone(s) are placed in ~/.sbt/1.0/staging/... where 1.0 is based on the sbt version number.
And, of course, don't forget to remove the reference to the other project if you have it in your libraryDependencies.
Here's the relevant part of my build.sbt file:
lazy val root = (project in file(".")).dependsOn(matchers)
lazy val matchers = RootProject(uri("git://github.com/rchillyard/Matchers#V1_0_5"))

Spark with IntelliJ or Eclipse

I am trying to setup IntelliJ for spark 2.11 but it is very daunting and after days I have not been able to compile a simple instruction such as with "spark.read.format" which is not found in main core and sql spark libraries.
I have seen a few posts on the subject but with none resolved. Does anyone have some experience with perhaps a working sample program I can start with?
Could it be that it would be easier with Eclipse?
Many thanks in advance for your answers,
EZ
build project in Intellij using with scala 2.11 and sbt 0.13: then ensure that your plugins.sbt contains as below:
logLevel := Level.Warn
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0")
then your build.sbt must contain as below:
scalaVersion := "2.11.8"
val sparkVersion = "2.1.0"
libraryDependencies += "org.apache.spark" %% "spark-core" % sparkVersion %"provided"
libraryDependencies += "org.apache.spark" %% "spark-sql" % sparkVersion %"provided"
Then write your code, click Terminal in Intellij and type sbt assembly: you can ship that jar to remote cluster, otherwise run from Intelij locally, let me know how it goes

Configuring Lift with xsbt-web-plugin: finding the main method class

I'm attempting to create 'from scratch' a Lift web application and running into some difficulty. I started with xsbt-web-plugin version 2.1, and managed to get it working with a simple Jetty servlet. Then I tried to integrate information from the Lift Cookbook.
This is my build.sbt:
organization := "ford.nathaniel"
name := "Lift From Scratch"
version := "0.1-SNAPSHOT"
scalaVersion := "2.11.7"
libraryDependencies += "javax.servlet" % "javax.servlet-api" % "3.0.1" % "provided"
enablePlugins(JettyPlugin)
libraryDependencies ++= {
val liftVersion = "3.0-M8"
Seq(
"net.liftweb" %% "lift-webkit" % liftVersion % "compile",
"org.eclipse.jetty" % "jetty-webapp" % "9.2.1.v20140609" % "container, test",
"org.eclipse.jetty" % "jetty-plus" % "9.2.1.v20140609" % "container, compile"
)
}
I have a (one line) project/plugins.sbt:
addSbtPlugin("com.earldouglas" % "xsbt-web-plugin" % "2.1.0")
I lifted the Boot.scala file directly from the above-linked Cookbook and placed it in src/main/scala/bootstrap, and similarly src/main/webapp/WEB-INF/web.xml. I can run sbt from the command line, and it loads cleanly, but when I try container:start...
> container:start
[info] starting server ...
[success] Total time: 0 s, completed Mar 3, 2016 9:52:25 PM
Error: Could not find or load main class
>
I am unclear on how sbt is meant to find Lift's main class. I looked through a lot of older versions of sbt configuration, which differ because earlier versions use earlier versions of the xsbt plugin. (Specifically, you see things like seq(webSettings :_*) - what does that even do?) I pulled recent versions of jetty-webapp and jetty-plus, so I don't think that's the problem. On the other hand, none of those configurations make clear how Lift figures out where Boot.scala is, and in turn I'm not sure how to write the configuration such that it knows how to bootstrap the framework.
One main difference between the two sources is that the new Jetty README configures the servlet like this:
libraryDependencies += "javax.servlet" % "javax.servlet-api" % "3.0.1" % "provided"
enablePlugins(JettyPlugin)
containerLibs in Jetty := Seq("org.eclipse.jetty" % "jetty-runner" % "9.2.1.v20140609" intransitive())
containerMain in Jetty := "org.eclipse.jetty.runner.Runner"
This seems to configure a SettingKey on the Jetty plugin, and point to the Jetty runner to kick off the servlet. However, given that I'm not running through a standard servlet, but through a filter configured to point to the Lift app in web.xml, this seemed like something I should remove.
Clearly I'm misunderstanding something. How do I diagnose why the xsbt-web-plugin is not picking up the Lift framework? It seems like either there is configuration-by-convention I can't find or I need to do something special with Jetty to point to the web.xml filter. Can anyone help clarify how to diagnose this, or elucidate how the two libraries should work together?
The following build.sbt ended up working for me when executed as jetty:start (rather than container:start). It appears that net.liftweb needs to be included as a libraryDependency rather than in the containerLibs, and that the Jetty plugin manages the container. I don't understand particularly, though, what the difference is or how to diagnose this in the future (ended up just trying a lot of different things).
Notably, though, the Boot.scala class of Lift is picked up automagically if the web.xml file is set up to use a Lift filter.
organization := "ford.nathaniel"
name := "Lift From Scratch"
version := "0.1-SNAPSHOT"
scalaVersion := "2.11.7"
libraryDependencies += "javax.servlet" % "javax.servlet-api" % "3.0.1" % "provided"
//logLevel := Level.Debug
enablePlugins(JettyPlugin)
libraryDependencies ++= {
val liftVersion = "3.0-M8"
Seq(
"net.liftweb" %% "lift-webkit" % liftVersion % "compile"
)
}
containerLibs in Jetty := {
val liftVersion = "3.0-M8"
Seq(
"org.eclipse.jetty" % "jetty-webapp" % "9.2.1.v20140609",
"org.eclipse.jetty" % "jetty-plus" % "9.2.1.v20140609",
"org.eclipse.jetty" % "jetty-runner" % "9.2.1.v20140609" intransitive()
)
}
containerMain in Jetty := "org.eclipse.jetty.runner.Runner"

How to toggle between project and library dependencies in SBT?

It's easy to declare managed library dependencies in SBT, eg
libraryDependencies ++= Seq(
"org.specs2" %% "specs2" % "1.12.2" % "test" ,
"junit" % "junit" % "4.7" % "test"
)
And while it's not as easy to declare project dependencies in SBT, I can do that too:
object RichMath extends Build {
lazy val myApp = Project("RichMath", file(".")) dependsOn(richUtil)
lazy val richUtil = RootProject(file("../RichUtil"))
}
But in practice, I typically want to change between project mode, where changes are immediately visible in upstream projects, and library mode, where I must publish changes to see them in dependent projects, as code matures.
Early in code-base's life, or whenever I'm wanting to make frequent changes across modules, I don't want the hassle of re-publishing just to see changes upstream. But in stable/mature code, I want to specify exactly what version's I'm depending upon.
It seems like SBT treats the two dependencies as completely different. Is there a more straight-forward way to switch between project- and library- dependencies than rewriting my build definition?
I have a few scenarios for my sbt scripts (tests, publishing, production). I start sbt from script (from bash, you may have other environment) with DO=TESTS sbt for example. This is my dynamic dependencies with regard of environment variable:
if (sys.env.contains("LOCAL_BUILD")) {
Seq[Project.Setting[_]](
unmanagedResourceDirectories in Compile <+= baseDirectory { _ / "src" / "main" / "scala" },
libraryDependencies ++= {
Seq(
"org.digimead" %% "digi-lib-slf4j" % "0.2.1-SNAPSHOT" % "test",
"org.digimead" %% "digi-lib-test" % "0.2.1-SNAPSHOT" % "test",
"org.scalatest" %% "scalatest" % "1.9" % "test"
)
}
)
} else {
Seq[Project.Setting[_]](
libraryDependencies ++= {
Seq(
"org.slf4j" % "slf4j-log4j12" % "1.7.1"
)
}
)
}
As you can see I may have different project settings with single .sbt definition controlled by one environment variable. The environment variable affect all project/subproject bunch.
It is true that the two types of dependencies are treated rather differently and it would be nice if they were not. The main obstacle is that sbt needs to know about all external projects before settings are loaded (for various reasons).
For now, the easiest solution is probably an environment variable or system property as described in another answer. Going forward, the following is very close to being possible in sbt, but still needs some more work:
Declare a dependency as usual
libraryDependencies += "org.example" % "rich-util" % "0.1"
Add the source dependency from the command line, overriding the normal dependency automatically in the process
$ sbt
> projects add ../RichUtil
The convention-based approach described in Setting up sbt environment to hack on multiple libraries at once is a special case and would be enabled by this working as well.

LIFT setup for web project: generate with mvn, and manage with sbt

I am new to LIFT, and I am trying to find a reliable instructions how to generate and manage LIFT web project with maven and sbt respectively. Can someone please direct me (or provide here) to the up to date instructions how to setup sbt for the maven generated project? From every post what I've red, it looks like the best setup for the LIFT projects: generate with mvn, manage with sbt. Will you agree? (I cannot generate LIFT/web project with sbt. Right? SBT is only good for managing it. Right? ) Every instructions I tried are out of date though. (I obviously can simply download and un-tar the archetype project, but I want to find a more fundamental approach for managing the environment ) Thanks.
While I'm using lift I don't need a maven at all, just SBT. So, it is very useful to read SBT Getting Started section. Also lift wiki contains some information. But be sure that you read material related to the proper SBT version. And finally, you can pay attention to my lift project template on github.
Good Luck with Lift! It's awesome ;)
By following question in you comment I put here some common config from my projects.
So, that is ./project/build.scala as alternative to ./build.sbt
import sbt._
import Keys._
import com.github.siasia._
import PluginKeys._
import WebPlugin._
import WebappPlugin._
object LiftProjectBuild extends Build {
override lazy val settings = super.settings ++ buildSettings
lazy val buildSettings = Seq(
organization := "com.yourorganization",
version := "0.1-SNAPSHOT",
scalaVersion := "2.9.1")
def yourWebSettings = webSettings ++ Seq(
// If you are using jrebel
scanDirectories in Compile := Nil
)
lazy val shade = Project(
id = "project-name",
base = file("."),
settings = defaultSettings ++ yourWebSettings)
lazy val defaultSettings = Defaults.defaultSettings ++ Seq(
name := "project-name",
resolvers ++= Seq(
"Typesafe Repo" at "http://repo.typesafe.com/typesafe/releases",
"Java.net Maven2 Repository" at "http://download.java.net/maven/2/"),
libraryDependencies ++= {
val liftVersion = "2.4-M5"
Seq(
"net.liftweb" %% "lift-webkit" % liftVersion % "compile",
"org.eclipse.jetty" % "jetty-webapp" % "7.5.4.v20111024" % "container",
"org.squeryl" %% "squeryl" % "0.9.5-SNAPSHOT" % "compile",
"ch.qos.logback" % "logback-classic" % "1.0.0" % "compile",
"org.scalatest" %% "scalatest" % "1.6.1" % "test",
"junit" % "junit" % "4.10" % "test")
},
// compile options
scalacOptions ++= Seq("-encoding", "UTF-8", "-deprecation", "-unchecked"),
javacOptions ++= Seq("-Xlint:unchecked", "-Xlint:deprecation"),
// show full stack traces
testOptions in Test += Tests.Argument("-oF")
)
}
./project/build.properties
#Project properties
sbt.version=0.11.1
./project/plugins.sbt
resolvers += Classpaths.typesafeResolver
addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse" % "1.5.0")
libraryDependencies <+= sbtVersion(v => "com.github.siasia" %% "xsbt-web-plugin" % (v+"-0.2.10"))
Having these three files are enough to configure sbt.
And of course you can run your application by calling container:start
Your mileage may vary, but my experience with sbt has been less than stellar (out-of-date docs, breakage on version changes, etc). The recent improvements to the eclipse scala IDE and corresponding maven and jrebel plugins make it a clear winner compared to using sbt.
If you follow the instructions, you'll get the ability to build/package at the command line, but superior support for eclipse features and fast development.
See the sample project setup at:
https://github.com/awkay/lift_squeryl_demo