Confused how to set up a multi-project sbt project - scala

I'm using sbt .13 here.
I have this so far:
import sbt._
import Keys._
import play.Project._
object ApplicationBuild extends Build {
val appVersion = "1.0"
resolvers += "local maven" at "/Users/blankman/.m2/repository/"
val commonDependencies = Seq()
val modelDependencies = Seq(
"com.typesafe.slick" %% "slick" % "2.0.1",
"org.slf4j" % "slf4j-nop" % "1.6.4"
)
val serviceDependencies = Seq(
"com.typesafe.slick" %% "slick" % "2.0.1",
"org.slf4j" % "slf4j-nop" % "1.6.4"
)
val webDependencies = Seq(
//"org.apache.tomcat" % "tomcat-jdbc" % "8.0.3",
"mysql" % "mysql-connector-java" % "5.1.30",
"com.typesafe.slick" %% "slick" % "2.0.1"
)
lazy val common = Project(
id = "app-common",
base = file("app-common"),
dependencies = commonDependencies
)
lazy val models = Project(
id = "app-models",
base = file("app-models"),
settings(modelDependencies: _*)
)
).dependsOn(common)
lazy val services = Project(
id = "app-services",
base = file("app-services"),
settings = Seq(
libraryDependencies ++= serviceDependencies
)
).dependsOn(models, common)
lazy val web = play.Project("app-web", appVersion, webDependencies,
path = file("app-web"))
.settings(playScalaSettings: _*)
.dependsOn(services)
}
This doesn't work. For example, if I go into:
project app-models
and try and compile, it says compile isn't valid or something.
I'm really confused how to set up a project. What is the correct way?
Looking at this slide #10 here http://jsuereth.com/scala/2013/06/11/effective-sbt.html it says I can do:
lazy val web = (
Project("app-models", file("app-models"))
settings(
libraryDependencies += modelDependencies
)
)
But when I do this I get an error also.
I basically have a few projects inside of sbt:
common
models
services
web (which is play)
models depends on commons
services depends on commons + models
web depends on services
Can someone help me get this to work?

There are a few issues I found in your build definition, but since you bought up Josh's Effective sbt talk, I think we should go whole hog on the style.
Effective sbt
Here are the files.
project/build.properties
sbt.version=0.13.2
project/play.sbt
val playVersion = "2.2.2"
resolvers += Resolver.typesafeRepo("releases")
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % playVersion)
project/commons.scala
import sbt._
import Keys._
object Commons {
val appVersion = "1.0"
val settings: Seq[Def.Setting[_]] = Seq(
version := appVersion,
resolvers += Opts.resolver.mavenLocalFile
)
}
project/dependencies.scala
import sbt._
import Keys._
object Dependencies {
val slickVersion = "2.0.1"
val slick = "com.typesafe.slick" %% "slick" % slickVersion
val slf4jVersion = "1.6.4"
val slf4jNop = "org.slf4j" % "slf4j-nop" % slf4jVersion
val mysqlConnectorVersion = "5.1.30"
val mysqlConnector = "mysql" % "mysql-connector-java" % mysqlConnectorVersion
val commonDependencies: Seq[ModuleID] = Seq(
slick,
slf4jNop
)
val modelDependencies: Seq[ModuleID] = Seq()
val serviceDependencies: Seq[ModuleID] = Seq()
val webDependencies: Seq[ModuleID] = Seq(
mysqlConnector
)
}
build.sbt
import play.Project._
import Dependencies._
lazy val appCommon = (project in file("app-common")).
settings(Commons.settings: _*).
settings(
libraryDependencies ++= commonDependencies
)
lazy val appModels = (project in file("app-models")).
settings(Commons.settings: _*).
settings(
libraryDependencies ++= modelDependencies
).
dependsOn(appCommon)
lazy val appServices = (project in file("app-services")).
settings(Commons.settings: _*).
settings(
libraryDependencies ++= serviceDependencies
).
dependsOn(appModels, appCommon)
lazy val appWeb = (project in file("app-web")).
settings(Commons.settings: _*).
settings(playScalaSettings: _*).
dependsOn(appServices)
notes
settings parameter vs settings method
For models and services, you're passing in settings sequence into Project(...) constructor, so the default settings are likely not loaded. You can pass in the default settings manually, or use settings(...) method on Project, which I would recommend.
lazy val appModels = (project in file("app-models")).
settings(
libraryDependencies ++= modelDependencies
).
dependsOn(appCommon)
Josh uses postfix notation using parenthesis, but I prefer using dot notation for this, so that's a slight deviation from the talk.
libraryDependencies ++=
As the above example, you have to pass modelDependencies to libraryDependencies. You had it calling directly into settings.
resolvers
The resolvers setting is not passed into anything, which likely is not correct.

Related

Read config file while generating Slick code via SBT

I am reading this sample which shows me how to generate source code using Slick-CodeGen
https://github.com/slick/slick-codegen-example/blob/master/build.sbt
And while this sample, is good, I want to modify it so that it reads the database config from application.conf using typesafe config.
Otherwise I will have to replicate the database connection configuration here and also in application.conf file.
Does anyone know, how can this sample be modified so that we can use the typesafe config to read the config values from application.conf?
Edit: Based on the suggestion below, I tried the following
I created a file called build.sbt in the project folder
libraryDependencies += "com.typesafe" % "config" % "1.3.1"
modified my main build.sbt file (in project root) as
val slickVersion = "3.1.1"
lazy val mainProject = Project(
id = "FooBar",
base=file("."),
settings = Defaults.coreDefaultSettings ++ Seq(
scalaVersion := "2.11.8",
libraryDependencies ++= Seq(
"com.typesafe.slick" %% "slick" % slickVersion,
"com.typesafe.slick" %% "slick-codegen" % slickVersion,
"mysql" % "mysql-connector-java" % "5.1.35",
"com.typesafe" % "config" % "1.3.1"
),
myConf := {
ConfigFactory.parseFile(new File("src/main/resources/application.conf"))
},
slick <<= slickCodeGenTask,
sourceGenerators in Compile <+= slickCodeGenTask
)
)
lazy val slick = TaskKey[Seq[File]]("gen-tables")
lazy val myConf = settingKey[Config]("The application properties")
lazy val slickCodeGenTask = (sourceManaged, dependencyClasspath in Compile, runner in Compile, streams) map {(dir, cp, r, s) =>
val outputDir = (dir / "slick").getPath
val username = myConf.value.getString("mysql.username")
val password = myConf.value.getString("mysql.password")
val port = myConf.value.getInt("mysql.port")
val db = myConf.value.getString("mysql.db")
val server = myConf.value.getString("mysql.server")
val url = s"jdbc:mysql://$server:$port/$db?username=$username&password=$password"
val jdbcDriver = myConf.value.getString("mysql.jdbcDriver")
val slickDriver = myConf.value.getString("mysql.slickDriver")
val pkg = "sql"
val fname = outputDir + "/db/Tables.scala"
toError(r.run("slick.codegen.SourceCodeGenerator", cp.files, Array(slickDriver, jdbcDriver, url, outputDir, pkg), s.log))
Seq(file(fname))
}
But it cannot resolve the Config and the ConfigFactory classes.
Declare a dependency on Typesafe Config in project/build.sbt:
libraryDependencies += "com.typesafe" % "config" % "1.3.1"
And define a setting holding your config file in build.sbt:
lazy val myConf = settingKey[Config]("The application properties")
myConf := {
ConfigFactory.parseFile(new File("src/main/resources/application.conf"))
}
Now you can use myConf.value.getString("xyz") to get hold of your configuration values in other tasks or settings.

SBT published jar corrupt

I am having a really strange problem:
I have a multi-module sbt project containing libraries that I publish locally.
Every once in a while the published jar is really a POM file!
Meaning I can open it with a text editor and see the POM for the library in question instead of the class files.
cleaning and rebuilding/republishing sometimes fixes it temporarily.
Also note that the jar in 'target/scala-2.11' is also a POM file.
Here is the Build.scala and sbt file for the libraries. (names are changed to protect the guilty :) due to company policies)
//////////////////Build.scala
import java.io._
import java.nio.file.{Paths, Files}
import sbt._
import xerial.sbt.Pack._
object Library extends Build {
lazy val rootProj = Project(id = "library", base = file(".")) aggregate(
projA,
projB,
projC,
projD
)
lazy val projA = Project(id = "projectA", base = file("ProjectA"))
lazy val projB = Project(id = "projectB", base = file("ProjectB"))
lazy val projC = Project(id = "projectC", base = file("ProjectC"))
.dependsOn(spatialMathProj)
lazy val projD = Project(id = "projectD", base = file("ProjectD"))
}
//////////////////build.sbt for projectD
name := "ProjectD"
version := "2.0-SNAPSHOT"
scalaVersion := "2.11.8"
publishArtifact in (Compile, packageDoc) := false
publishArtifact in (Compile, packageSrc) := false
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.4.2"
libraryDependencies += "com.typesafe.akka" %% "akka-remote" % "2.4.2"
libraryDependencies += "net.sf.jung" % "jung2" % "2.0.1"
libraryDependencies += "net.sf.jung" % "jung-graph-impl" % "2.0.1"
libraryDependencies += "net.sf.jung" % "jung-visualization" % "2.0.1"
libraryDependencies += "com.plexsys" % "api_2.11" % "2.0-SNAPSHOT"
Has anyone else seen this?
Why is this happening?
I am using sbt version 0.13.7.
Thanks for any help you can offer.

Why does SBT compile just the last subproject?

I'm getting crazy, really :-(
Given the following SBT build file...
import sbt._
import sbt.Keys._
object BrixBuild extends Build {
lazy val buildSettings = Project.defaultSettings ++ Seq(
organization := "com.mycompany",
version := "0.1-SNAPSHOT",
scalaVersion := "2.10.0-RC3",
scalacOptions in Compile ++= Seq("-encoding", "UTF-8", "-deprecation", "-feature", "-unchecked"),
resolvers += Resolvers.typesafe,
target := file("target")
)
lazy val util = Project(
id = "brix-util",
base = file("brix-util"),
settings = buildSettings ++ Seq(
libraryDependencies ++= Dependencies.util
)
)
lazy val slick = Project(
id = "brix-slick",
base = file("brix-slick"),
settings = buildSettings ++ Seq(
libraryDependencies ++= Dependencies.slick
)
)
}
object Resolvers {
val typesafe = "Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases"
}
object Dependencies {
private object Compile {
val commonsCodec = "commons-codec" % "commons-codec" % "1.7"
val slick = "com.typesafe" %% "slick" % "1.0.0-RC1"
}
private object Test {
val specs2 = "org.specs2" %% "specs2" % "1.12.3" % "test"
val slf4j = "org.slf4j" % "slf4j-nop" % "1.6.4" % "test"
val h2 = "com.h2database" % "h2" % "1.3.166" % "test"
}
val util = Seq(Compile.commonsCodec, Test.specs2, Test.slf4j)
val slick = Seq(Compile.slick, Test.specs2, Test.slf4j, Test.h2)
}
... only the last subproject get compiled. Why? Am I missing something?
Any help would be really appreciated. Tx.
You should define a "parent" project that aggregates the subprojects. For an example see https://github.com/typesafehub/scalalogging/blob/master/project/Build.scala.

How can I add jars from more than one unmanaged directory in an SBT .scala project configuration

I'm trying to get SBT to build a project that could have more than one unmanaged directory. If I had a single directory, I could easily do it like this:
unmanagedBase := file( "custom-libs" ).getAbsoluteFile
But since I have two directories with unmanaged jars, I need to be able to add them all. I have found some information in here, but still doesn't seem useful for my full .scala file build.
I have created a simple project that shows the issue in here. And below is my Build.scala file.
UPDATE
I've got some help form the sbt-users list and have been able to define the unmanaged-jars correctly, but the code still doesn't compile (but sbt show unmanaged-jars shows the files correctly).
import sbt._
import com.github.siasia._
import PluginKeys._
import Keys._
object Build extends sbt.Build {
import Dependencies._
val unmanagedListing = unmanagedJars := {
Dependencies.listUnmanaged( file(".").getAbsoluteFile )
}
lazy val myProject = Project("spray-template", file("."))
.settings(WebPlugin.webSettings: _*)
.settings(port in config("container") := 8080)
.settings(
organization := "com.example",
version := "0.9.0-RC1",
scalaVersion := "2.9.1",
scalacOptions := Seq("-deprecation", "-encoding", "utf8"),
resolvers ++= Dependencies.resolutionRepos,
libraryDependencies ++= Seq(
Compile.akkaActor,
Compile.sprayServer,
Test.specs2,
Container.jettyWebApp,
Container.akkaSlf4j,
Container.slf4j,
Container.logback
),
unmanagedListing
)
}
object Dependencies {
val resolutionRepos = Seq(
ScalaToolsSnapshots,
"Typesafe repo" at "http://repo.typesafe.com/typesafe/releases/",
"spray repo" at "http://repo.spray.cc/"
)
def listUnmanaged( base : RichFile ) : Keys.Classpath = {
val baseDirectories = (base / "custom-libs") +++ ( base / "custom-libs2" )
(baseDirectories ** "*.jar").classpath
}
object V {
val akka = "1.3"
val spray = "0.9.0-RC1"
val specs2 = "1.7.1"
val jetty = "8.1.0.v20120127"
val slf4j = "1.6.4"
val logback = "1.0.0"
}
object Compile {
val akkaActor = "se.scalablesolutions.akka" % "akka-actor" % V.akka % "compile"
val sprayServer = "cc.spray" % "spray-server" % V.spray % "compile"
}
object Test {
val specs2 = "org.specs2" %% "specs2" % V.specs2 % "test"
}
object Container {
val jettyWebApp = "org.eclipse.jetty" % "jetty-webapp" % V.jetty % "container"
val akkaSlf4j = "se.scalablesolutions.akka" % "akka-slf4j" % V.akka
val slf4j = "org.slf4j" % "slf4j-api" % V.slf4j
val logback = "ch.qos.logback" % "logback-classic" % V.logback
}
}
I just post the fragment from my build.sbt file, using sbt 0.11.x. It could probably be refactored a bit.
unmanagedJars in Compile <++= baseDirectory map { base =>
val libs = base / "lib"
val dirs = (libs / "batik") +++ (libs / "libtw") +++ (libs / "kiama")
(dirs ** "*.jar").classpath
}
You can add additional paths to the list of folders to scan for unmanaged dependencies. For example, to look in a folder called "config" in addition to "lib" for the run task, you can add the following. For the compile task change Runtime to Compile.
unmanagedClasspath in Runtime <+= (baseDirectory) map {
bd => Attributed.blank(bd / "config")
}
As answered by Eugene Vigdorchik, what made it work as the following code:
import sbt._
import com.github.siasia._
import PluginKeys._
import Keys._
object Build extends sbt.Build {
import Dependencies._
var unmanagedListing = unmanagedJars in Compile := {
Dependencies.listUnmanaged( file(".").getAbsoluteFile )
}
lazy val myProject = Project("spray-template", file("."))
.settings(WebPlugin.webSettings: _*)
.settings(port in config("container") := 8080)
.settings(
organization := "com.example",
version := "0.9.0-RC1",
scalaVersion := "2.9.1",
scalacOptions := Seq("-deprecation", "-encoding", "utf8"),
resolvers ++= Dependencies.resolutionRepos,
libraryDependencies ++= Seq(
C.akkaActor,
C.sprayServer,
Test.specs2,
Container.jettyWebApp,
Container.akkaSlf4j,
Container.slf4j,
Container.logback
),
unmanagedListing
)
}
object Dependencies {
val resolutionRepos = Seq(
ScalaToolsSnapshots,
"Typesafe repo" at "http://repo.typesafe.com/typesafe/releases/",
"spray repo" at "http://repo.spray.cc/"
)
def listUnmanaged( base : RichFile ) : Keys.Classpath = {
val baseDirectories = (base / "custom-libs") +++ ( base / "custom-libs2" )
(baseDirectories ** "*.jar").classpath
}
object V {
val akka = "1.3"
val spray = "0.9.0-RC1"
val specs2 = "1.7.1"
val jetty = "8.1.0.v20120127"
val slf4j = "1.6.4"
val logback = "1.0.0"
}
object C {
val akkaActor = "se.scalablesolutions.akka" % "akka-actor" % V.akka % "compile"
val sprayServer = "cc.spray" % "spray-server" % V.spray % "compile"
}
object Test {
val specs2 = "org.specs2" %% "specs2" % V.specs2 % "test"
}
object Container {
val jettyWebApp = "org.eclipse.jetty" % "jetty-webapp" % V.jetty % "container"
val akkaSlf4j = "se.scalablesolutions.akka" % "akka-slf4j" % V.akka
val slf4j = "org.slf4j" % "slf4j-api" % V.slf4j
val logback = "ch.qos.logback" % "logback-classic" % V.logback
}
}
Source repo with the full example available on Github.
Here's a general solution to recursive loading of unmanaged JARs (for sbt 0.13.x): https://stackoverflow.com/a/29357699/1348306

sbt web plugin: Not a valid key: jetty-run (similar: jetty-port, jetty-context, run)

I'm trying to set up a scala sbt project with the lift web framework. I'm using
scala 2.9.0-1
sbt 0.10.1
lift 2.3
xsbt-web-plugin 0.1.1 (which is only on scala 2.8.1, see end of question)
(quite recent versions I know).
I followed http://d.hatena.ne.jp/k4200/20110711/1310354698 and https://github.com/siasia/xsbt-web-plugin/blob/master/README.md to obtain the following sbt configuration files:
project/build.properties
sbt.version=0.10.1
project/plugins/build.sbt
resolvers += "Web plugin repo" at "http://siasia.github.com/maven2"
libraryDependencies <+= sbtVersion(v => "com.github.siasia" % "xsbt-web-plugin_2.8.1" % ("0.1.1-"+v))
project/Build.scala
import sbt._
import Keys._
object BuildSettings {
val buildOrganization = "xbaz"
val buildScalaVersion = "2.9.0-1"
val buildVersion = "0.0.1"
val buildSettings = Defaults.defaultSettings ++ Seq (
organization := buildOrganization,
scalaVersion := buildScalaVersion,
version := buildVersion)
}
object Resolvers {
val webPluginRepo = "Web plugin repo" at "http://siasia.github.com/maven2"
val jettyRepo = "Jetty Repo" at "http://repo1.maven.org/maven2/org/mortbay/jetty"
}
object Dependencies {
// web plugin
val webPluginDeps = Seq(
"org.mortbay.jetty" % "jetty" % "6.1.26" % "jetty", // The last part is "jetty" not "test".
"javax.servlet" % "servlet-api" % "2.5" % "provided->default"
)
val liftDeps = {
val liftVersion = "2.3" // I'll switch to 2.3 soon!
Seq(
"net.liftweb" % "lift-webkit_2.8.1" % liftVersion % "compile->default",
"net.liftweb" % "lift-mapper_2.8.1" % liftVersion % "compile->default"
)
}
val scalaTest = "org.scalatest" % "scalatest_2.9.0" % "1.6.1" % "test"
val apacheHttpClient = "org.apache.httpcomponents" % "httpclient" % "4.1.1"
val apacheHttpCore = "org.apache.httpcomponents" % "httpcore" % "4.1.1"
// Logging
lazy val grizzled = "org.clapper" % "grizzled-slf4j_2.8.1" % "0.5"
lazy val junit = "junit" % "junit" % "4.8" % "test"
lazy val logback_core = "ch.qos.logback" % "logback-core" % "0.9.24" % "compile" //LGPL 2.1
lazy val logback_classic = "ch.qos.logback" % "logback-classic" % "0.9.24" % "compile" //LGPL 2.1
lazy val log4j_over_slf4j = "org.slf4j" % "log4j-over-slf4j" % "1.6.1"
val logDeps = Seq(grizzled, log4j_over_slf4j, logback_core, logback_classic)
}
object MyBuild extends Build {
import com.github.siasia.WebPlugin._ // web plugin
import BuildSettings._
import Dependencies._
import Resolvers._
//End dependencies
lazy val root = Project("root", file(".") , settings = buildSettings ++
Seq( name := "foo")
) aggregate(core, cli, web)
// mainClass:= Some("Main"))
lazy val core : Project = Project("core", file("core"), delegates = root :: Nil, settings = buildSettings ++
Seq(
name := "foo-core",
libraryDependencies ++= logDeps ++ Seq(scalaTest, apacheHttpClient, apacheHttpCore)
)
)
lazy val cli: Project = Project("cli", file("cli"), settings = buildSettings ++
Seq(
name := "foo-cli",
libraryDependencies ++= logDeps ++ Seq(apacheHttpClient),
fork in run := true,
javaOptions in run += "-Djava.library.path=/home/jolivier/Projets/asknow/lib/jnotify-lib-0.93"
)) dependsOn(core) settings(
)
lazy val web: Project = Project("web", file("web"), settings = buildSettings ++
Seq (resolvers := Seq(webPluginRepo, jettyRepo),
name := "foo-http",
libraryDependencies ++= logDeps ++ webPluginDeps ++ liftDeps
) ++
webSettings
) dependsOn(core)
}
When I try sbt jetty-run I get the following error message:
[error] Not a valid command: jetty-run
[error] Not a valid project ID: jetty-run
[error] Not a valid configuration: jetty-run
[error] Not a valid key: jetty-run (similar: jetty-port, jetty-context, run)
[error] jetty-run
[error]
So I noticed that some jetty-* commands do exist, but not the one I want, so I printed webSettings which is supposed to contain all these new settings and it contains jetty-context and jetty-port, as well as jetty-configuration and others, but not jetty-run :s.
What did I go wrong to not have jetty-run?
I tried switching to scala-2.8.1 since the web plugin is currently only on scala 2.8.1, by changing my buildScalaVersion variable but that didn't change anything. Do you have any idea?
Thanks in advance for your help
Tasks are aggregated; commands are not.
jetty-run is a command. It is only available in the context of the sub-project with the web plugin settings.
> project web
> jetty-run
Once it is running, you can use the prepare-webapp task to redeploy the webapp. This can be run from the context of the root project, because it aggregates the web project.