Play sub-projects do not compile - scala

Here is the layout of my multi-project Play 2.2 application - I'm still trying to convert to build.sbt:
myApp
+ app
+ build.sbt
+ conf
| + routes
+ project
| + build.properties
| + Build.scala
| + plugin.sbt
+ modules
+ myModule
+ app
+ build.sbt
+ conf
+ routes
myApp/build.sbt:
name := "myApp"
version := "1.0-SNAPSHOT"
organization := "com.mydomain"
lazy val myModule = project.in(file("modules/myModule"))
lazy val main = project.in(file(".")).dependsOn(myModule).aggregate(myModule)
play.Project.playScalaSettings
resolvers ++= Seq(
Resolvers.typesafe,
Resolvers.sonatype
)
myApp/projects/Build.scala:
import sbt._
object Resolvers {
val sonatype = Resolver.sonatypeRepo("snapshots")
val typesafe = "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"
}
myApp/modules/myModule/build.sbt:
name := "myModule"
libraryDependencies ++= Seq(
"com.typesafe.play" %% "play" % "2.2.1" % "provided",
"org.reactivemongo" %% "reactivemongo" % "0.10.0-SNAPSHOT",
"org.reactivemongo" %% "play2-reactivemongo" % "0.10.0-SNAPSHOT"
)
resolvers ++= Seq(
Resolvers.typesafe,
Resolvers.sonatype
)
I'm facing with two problems when trying to compile the project above:
1) Even if only the sub-project has dependencies (the main project is just a container for many sub-projects), I have to specify the resolvers also in the main build file myApp/build.sbt; if I don't, the project doesn't compile. This is not a blocking problem.. but I'd like to understand why.
2) Then, as soon as I try to compile the project, I always get the following error:
[error] /home/j3d/Projects/myApp/conf/routes:9: not found: value myModule
[error] -> /myModule myModule.Routes
[error] /home/j3d/Projects/myApp/conf/routes: not found: value myModule
[error] /home/j3d/Projects/myApp/conf/routes:12: not found: value myModule
[error] GET /assets/*file controllers.Assets.at(path="/public", file)
[error] /home/j3d/Projects/myApp/conf/routes:9: not found: value handler
[error] -> /myModule myModule.Routes
[error] four errors found
[error] (main/compile:compile) Compilation failed
[error] Total time: 12 s, completed Dec 1, 2013 6:34:55 PM
Here is myApp/conf/routes...
GET / controllers.Application.index
-> /myModule myModule.Routes
GET /assets/*file controllers.Assets.at(path="/public", file)
... and finally here is myApp/modules/myModule/conf/myModule.routes:
GET /myModule/greetings controllers.myModule.Greetings.hello
Am I missing something?

Figured out how to make it work and here below is my solution. First of all I've defined default settings and resolvers for all projects [myApp/project/Build.scala]:
import sbt._
import Keys._
object ApplicationBuild extends Build {
val defaultResolvers = Seq(
Resolver.sonatypeRepo("snapshots"),
"Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"
)
val defaultSettings = Defaults.defaultSettings ++ Seq(
scalacOptions += "-language:reflectiveCalls",
resolvers ++= defaultResolvers
)
}
Then I've just imported ApplicationBuild.defaultSettings into every build.sbt. Here is the main project build file [myApp/build.sbt]...
name := "myApp"
version := "1.0-SNAPSHOT"
lazy val auth = project.in(file("modules/myModule"))
lazy val main = project.in(file(".")).dependsOn(myModule).aggregate(myModule)
ApplicationBuild.defaultSettings
playScalaSettings
... and here the sub-project build file [myApp/modules/myModule/build.sbt]:
name := "myModule"
ApplicationBuild.defaultSettings
playScalaSettings
libraryDependencies ++= Seq(
"org.reactivemongo" %% "play2-reactivemongo" % "0.10.0-SNAPSHOT",
"org.reactivemongo" %% "reactivemongo" % "0.10.0-SNAPSHOT"
)
It just works and hope it helps ;-)

Related

Understanding build.sbt with sbt-spark-package plugin

I am new the scala and SBT build files. From the introductory tutorials adding spark dependencies to a scala project should be straight-forward via the sbt-spark-package plugin but I am getting the following error:
[error] (run-main-0) java.lang.NoClassDefFoundError: org/apache/spark/SparkContext
Please provide resources to learn more about what could be driving error as I want to understand process more thoroughly.
CODE:
trait SparkSessionWrapper {
lazy val spark: SparkSession = {
SparkSession
.builder()
.master("local")
.appName("spark citation graph")
.getOrCreate()
}
val sc = spark.sparkContext
}
import org.apache.spark.graphx.GraphLoader
object Test extends SparkSessionWrapper {
def main(args: Array[String]) {
println("Testing, testing, testing, testing...")
var filePath = "Desktop/citations.txt"
val citeGraph = GraphLoader.edgeListFile(sc, filepath)
println(citeGraph.vertices.take(1))
}
}
plugins.sbt
resolvers += "bintray-spark-packages" at "https://dl.bintray.com/spark-packages/maven/"
addSbtPlugin("org.spark-packages" % "sbt-spark-package" % "0.2.6")
build.sbt -- WORKING. Why does libraryDependencies run/work ?
spName := "yewno/citation_graph"
version := "0.1"
scalaVersion := "2.11.12"
sparkVersion := "2.2.0"
sparkComponents ++= Seq("core", "sql", "graphx")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.2.0",
"org.apache.spark" %% "spark-sql" % "2.2.0",
"org.apache.spark" %% "spark-graphx" % "2.2.0"
)
build.sbt -- NOT WORKING. Would expect this to compile & run correctly
spName := "yewno/citation_graph"
version := "0.1"
scalaVersion := "2.11.12"
sparkVersion := "2.2.0"
sparkComponents ++= Seq("core", "sql", "graphx")
Bonus for explanation + links to resources to learn more about SBT build process, jar files, and anything else that can help me get up to speed!
sbt-spark-package plugin provides dependencies in provided scope:
sparkComponentSet.map { component =>
"org.apache.spark" %% s"spark-$component" % sparkVersion.value % "provided"
}.toSeq
We can confirm this by running show libraryDependencies from sbt:
[info] * org.scala-lang:scala-library:2.11.12
[info] * org.apache.spark:spark-core:2.2.0:provided
[info] * org.apache.spark:spark-sql:2.2.0:provided
[info] * org.apache.spark:spark-graphx:2.2.0:provided
provided scope means:
The dependency will be part of compilation and test, but excluded from
the runtime.
Thus sbt run throws java.lang.NoClassDefFoundError: org/apache/spark/SparkContext
If we really want to include provided dependencies on run classpath then #douglaz suggests:
run in Compile := Defaults.runTask(fullClasspath in Compile, mainClass in (Compile, run), runner in (Compile, run)).evaluated

sbt cross project, shared dependencies for test example

I have a small project.
Where I have the following problem:
scalaTest needs to be added to all three dependency project (client, server, shared), otherwise the scalatest library is not accessible from all projects.
In other words, if I write
val jvmDependencies = Def.setting(Seq(
"org.scalaz" %% "scalaz-core" % "7.2.8"
)++scalaTest)
then things work fine.
But if I don't write ++scalaTest into each three dependencies then it fails like this:
> test
[info] Compiling 1 Scala source to /Users/joco/tmp3/server/target/scala-2.11/test-classes...
[error] /Users/joco/tmp3/server/src/test/scala/Test.scala:1: object specs2 is not a member of package org
[error] import org.specs2.mutable.Specification
[error] ^
[error] /Users/joco/tmp3/server/src/test/scala/Test.scala:3: not found: type Specification
[error] class Test extends Specification {
[error] ^
[error] /Users/joco/tmp3/server/src/test/scala/Test.scala:5: value should is not a member of String
[error] "Test" should {
[error] ^
[error] /Users/joco/tmp3/server/src/test/scala/Test.scala:6: value in is not a member of String
[error] "one is one" in {
[error] ^
[error] /Users/joco/tmp3/server/src/test/scala/Test.scala:8: value === is not a member of Int
[error] 1===one
[error] ^
[error] 5 errors found
[error] (server/test:compileIncremental) Compilation failed
[error] Total time: 4 s, completed Mar 18, 2017 1:56:54 PM
However for production(not test) code everything works just fine: I don't have to add 3 times the same dependencies (in this example autowire) to all three projects if I want to use a library in all three projects, it is enough to add it to only the shared project and then I can use that library from all three projects.
For test code, however, as I mentioned above, currently I have to add the same library dependency (scalaTest - below) to all three projects.
Question: Is there a way to avoid this ?
Settings.scala:
import org.scalajs.sbtplugin.ScalaJSPlugin.autoImport._
import sbt.Keys._
import sbt._
object Settings {
val scalacOptions = Seq(
"-Xlint",
"-unchecked",
"-deprecation",
"-feature",
"-Yrangepos"
)
object versions {
val scala = "2.11.8"
}
val scalaTest=Seq(
"org.scalatest" %% "scalatest" % "3.0.1" % "test",
"org.specs2" %% "specs2" % "3.7" % "test")
val sharedDependencies = Def.setting(Seq(
"com.lihaoyi" %%% "autowire" % "0.2.6"
)++scalaTest)
val jvmDependencies = Def.setting(Seq(
"org.scalaz" %% "scalaz-core" % "7.2.8"
))
/** Dependencies only used by the JS project (note the use of %%% instead of %%) */
val scalajsDependencies = Def.setting(Seq(
"org.scala-js" %%% "scalajs-dom" % "0.9.1"
)++scalaTest)
}
build.sbt:
import sbt.Keys._
import sbt.Project.projectToRef
import webscalajs.SourceMappings
lazy val shared = (crossProject.crossType(CrossType.Pure) in file("shared")) .settings(
scalaVersion := Settings.versions.scala,
libraryDependencies ++= Settings.sharedDependencies.value,
addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full)
) .jsConfigure(_ enablePlugins ScalaJSWeb)
lazy val sharedJVM = shared.jvm.settings(name := "sharedJVM")
lazy val sharedJS = shared.js.settings(name := "sharedJS")
lazy val elideOptions = settingKey[Seq[String]]("Set limit for elidable functions")
lazy val client: Project = (project in file("client"))
.settings(
scalaVersion := Settings.versions.scala,
scalacOptions ++= Settings.scalacOptions,
libraryDependencies ++= Settings.scalajsDependencies.value,
testFrameworks += new TestFramework("utest.runner.Framework")
)
.enablePlugins(ScalaJSPlugin)
.disablePlugins(RevolverPlugin)
.dependsOn(sharedJS)
lazy val clients = Seq(client)
lazy val server = (project in file("server")) .settings(
scalaVersion := Settings.versions.scala,
scalacOptions ++= Settings.scalacOptions,
libraryDependencies ++= Settings.jvmDependencies.value
)
.enablePlugins(SbtLess,SbtWeb)
.aggregate(clients.map(projectToRef): _*)
.dependsOn(sharedJVM)
onLoad in Global := (Command.process("project server", _: State)) compose (onLoad in Global).value
fork in run := true
cancelable in Global := true
For test code, however, as I mentioned above, currently I have to add the same library dependency (scalaTest - below) to all three projects.
That is expected. test dependencies are not inherited along dependency chains. That makes sense, because you don't want to depend on JUnit just because you depend on a library that happens to be tested using JUnit.
Although yes, that calls for a bit of duplication when you have several projects in the same build, all using the same testing framework. This is why we often find some commonSettings that are added to all projects of an sbt build. This is also where we typically put things like organization, scalaVersion, and many other settings that usually apply to all projects inside one build.

"object github is not a member of package com" for WeKanban2 example project in Scala in action book

I'm new in Scala so started from reading "Scala in Action" book 2013 year. But I cannot build plain sbt WeKanban2 project. I use newer version scala and sbt then in example.
I get error when run sbt:
error: object github is not a member of package com import com.github.siasia.WebPlugin._
my build.properties:
//was sbt.version=0.12.0
sbt.version=0.13.7
plugins.sbt
// was libraryDependencies <+= sbtVersion(v => "com.github.siasia" %% "xsbt-eb-plugin" (v+"-0.2.11.1"))
addSbtPlugin("com.earldouglas" % "xsbt-web-plugin" % "0.7.0")
build.sbt
import com.github.siasia.WebPlugin._
name := "weKanban"
organization := "scalainaction"
version := "0.2"
//was: scalaVersion := "2.10.0"
scalaVersion := "2.11.4"
scalacOptions ++= Seq("-unchecked", "-deprecation")
resolvers ++= Seq(
"Scala-Tools Maven2 Releases Repository" at "http://scala-tools.org/repo-releases",
"Scala-Tools Maven2 Snapshots Repository" at "http://scala-tools.org/repo-snapshots"
)
libraryDependencies ++= Seq(
"org.scalaz" %% "scalaz-core" % scalazVersion,
"org.scalaz" %% "scalaz-http" % scalazVersion,
"org.eclipse.jetty" % "jetty-servlet" % jettyVersion % "container",
"org.eclipse.jetty" % "jetty-webapp" % jettyVersion % "test, container",
"org.eclipse.jetty" % "jetty-server" % jettyVersion % "container",
"com.h2database" % "h2" % "1.2.137",
"org.squeryl" % "squeryl_2.10" % "0.9.5-6"
)
seq(webSettings :_*)
build.scala
import sbt._
import Keys._
object H2TaskManager {
var process: Option[Process] = None
lazy val H2 = config("h2") extend(Compile)
val startH2 = TaskKey[Unit]("start", "Starts H2 database")
val startH2Task = startH2 in H2 <<= (fullClasspath in Compile) map { cp =>
startDatabase(cp.map(_.data).map(_.getAbsolutePath()).filter(_.contains("h2database")))}
def startDatabase(paths: Seq[String]) = {
process match {
case None =>
val cp = paths.mkString(System.getProperty("path.seperator"))
val command = "java -cp " + cp + " org.h2.tools.Server"
println("Starting Database with command: " + command)
process = Some(Process(command).run())
println("Database started ! ")
case Some(_) =>
println("H2 Database already started")
}
}
val stopH2 = TaskKey[Unit]("stop", "Stops H2 database")
val stopH2Task = stopH2 in H2 :={
process match {
case None => println("Database already stopped")
case Some(_) =>
println("Stopping database...")
process.foreach{_.destroy()}
process = None
println("Database stopped...")
}
}
}
object MainBuild extends Build {
import H2TaskManager._
lazy val scalazVersion = "6.0.3"
lazy val jettyVersion = "7.3.0.v20110203"
lazy val wekanban = Project(
"wekanban",
file(".")) settings(startH2Task, stopH2Task)}
What should I do to fix this error? What should I do to start app from Idea 14 in debug mode?
Your import
import com.github.siasia.WebPlugin._
failed to find path github and below. In your plugins.sbt you wrote:
// was libraryDependencies <+= sbtVersion(v => "com.github.siasia" %% "xsbt-eb-plugin" (v+"-0.2.11.1"))
addSbtPlugin("com.earldouglas" % "xsbt-web-plugin" % "0.7.0")
The github-path probably existed in the outcommented com.github.siasia.xsbt-eb-plugin. However the com.earldouglas.xsbt-web-plugin have only the path com.earldouglas.xwp. You should check out if any of the classes there (https://github.com/earldouglas/xsbt-web-plugin/tree/master/src/main/scala) resembles what you need to import.
Probably not related exactly to this package but similar. The library was present in sbt and everything looked ok. But I still was unable to build the project so I went to my $HOME/.ivy2/cache located the dependency directory and deleted it, then went back the $PROJECT_HOME/ and executed sbt build

SBT setup for sbt-scalabuff

Can someone give me a minimal working sbt setup for sbt-scalabuff? The information that's out there seems incomplete. I'm currently trying to use addSbtPlugin("com.github.sbt" % "sbt-scalabuff" % "0.2"), but I get sbt.ResolveException: unresolved dependency: net.sandrogrzicic#scalabuff-runtime_2.9.2;1.3.6: not found. I guess I'm missing a repository.
Why is it using 2.9.2, though? I have scalaVersion := 2.10.3.
build.sbt
organization := "com.confabulous"
name := "protobuf"
version := "0.0.1-SNAPSHOT"
scalaVersion := "2.10.4"
scalacOptions += "-deprecation"
resolvers ++= Seq(
"sonatype releases" at "https://oss.sonatype.org/content/repositories/releas
"sonatype snapshots" at "https://oss.sonatype.org/content/repositories/snapsh
"typesafe repo" at "http://repo.typesafe.com/typesafe/releases/"
)
libraryDependencies += "net.sandrogrzicic" %% "scalabuff-runtime" % "1.3.6"
plugins/plugins.sbt
addSbtPlugin("com.github.sbt" % "sbt-scalabuff" % "0.2")
project/Build.scala
import sbt._
import scalabuff.ScalaBuffPlugin._
object build extends Build {
lazy val root = Project("main", file("."), settings = Defaults.defaultSetting
}
Output
$ sbt compile
Loading /usr/share/sbt/bin/sbt-launch-lib.bash
[info] Loading project definition from /home/dan/projects/confabulous/protobuf/project
[info] Updating {file:/home/dan/projects/confabulous/protobuf/project/}default-6a3ff1...
[info] Resolving org.scala-sbt#precompiled-2_10_1;0.12.4 ...
[info] Done updating.
[info] Set current project to protobuf (in build file:/home/dan/projects/confabulous/protobuf/)
[info] Compiling 1 Scala source to /home/dan/projects/confabulous/protobuf/target/scala-2.10/classes...
[error] /home/dan/projects/confabulous/protobuf/target/scala-2.10/src_managed/scala/com/confabulous/protobuf/ConfabulousProtobuf.scala:11: not found: value net
[error] with net.sandrogrzicic.scalabuff.Message[Pair] {
[error] ^
[error] /home/dan/projects/confabulous/protobuf/target/scala-2.10/src_managed/scala/com/confabulous/protobuf/ConfabulousProtobuf.scala:76: not found: value net
[error] with net.sandrogrzicic.scalabuff.Message[Notice] {
[error] ^
Add https://dl.bintray.com/actor/maven to your SBT resolver
resolvers ++= Seq("sonatype releases" at "https://oss.sonatype.org/content/repositories/releas
"sonatype snapshots" at "https://oss.sonatype.org/content/repositories/snapsh
"typesafe repo" at "http://repo.typesafe.com/typesafe/releases/"
"bintrayRepo" at "https://dl.bintray.com/actor/maven"
)

sbt web plugin: Not a valid key: jetty-run (similar: jetty-port, jetty-context, run)

I'm trying to set up a scala sbt project with the lift web framework. I'm using
scala 2.9.0-1
sbt 0.10.1
lift 2.3
xsbt-web-plugin 0.1.1 (which is only on scala 2.8.1, see end of question)
(quite recent versions I know).
I followed http://d.hatena.ne.jp/k4200/20110711/1310354698 and https://github.com/siasia/xsbt-web-plugin/blob/master/README.md to obtain the following sbt configuration files:
project/build.properties
sbt.version=0.10.1
project/plugins/build.sbt
resolvers += "Web plugin repo" at "http://siasia.github.com/maven2"
libraryDependencies <+= sbtVersion(v => "com.github.siasia" % "xsbt-web-plugin_2.8.1" % ("0.1.1-"+v))
project/Build.scala
import sbt._
import Keys._
object BuildSettings {
val buildOrganization = "xbaz"
val buildScalaVersion = "2.9.0-1"
val buildVersion = "0.0.1"
val buildSettings = Defaults.defaultSettings ++ Seq (
organization := buildOrganization,
scalaVersion := buildScalaVersion,
version := buildVersion)
}
object Resolvers {
val webPluginRepo = "Web plugin repo" at "http://siasia.github.com/maven2"
val jettyRepo = "Jetty Repo" at "http://repo1.maven.org/maven2/org/mortbay/jetty"
}
object Dependencies {
// web plugin
val webPluginDeps = Seq(
"org.mortbay.jetty" % "jetty" % "6.1.26" % "jetty", // The last part is "jetty" not "test".
"javax.servlet" % "servlet-api" % "2.5" % "provided->default"
)
val liftDeps = {
val liftVersion = "2.3" // I'll switch to 2.3 soon!
Seq(
"net.liftweb" % "lift-webkit_2.8.1" % liftVersion % "compile->default",
"net.liftweb" % "lift-mapper_2.8.1" % liftVersion % "compile->default"
)
}
val scalaTest = "org.scalatest" % "scalatest_2.9.0" % "1.6.1" % "test"
val apacheHttpClient = "org.apache.httpcomponents" % "httpclient" % "4.1.1"
val apacheHttpCore = "org.apache.httpcomponents" % "httpcore" % "4.1.1"
// Logging
lazy val grizzled = "org.clapper" % "grizzled-slf4j_2.8.1" % "0.5"
lazy val junit = "junit" % "junit" % "4.8" % "test"
lazy val logback_core = "ch.qos.logback" % "logback-core" % "0.9.24" % "compile" //LGPL 2.1
lazy val logback_classic = "ch.qos.logback" % "logback-classic" % "0.9.24" % "compile" //LGPL 2.1
lazy val log4j_over_slf4j = "org.slf4j" % "log4j-over-slf4j" % "1.6.1"
val logDeps = Seq(grizzled, log4j_over_slf4j, logback_core, logback_classic)
}
object MyBuild extends Build {
import com.github.siasia.WebPlugin._ // web plugin
import BuildSettings._
import Dependencies._
import Resolvers._
//End dependencies
lazy val root = Project("root", file(".") , settings = buildSettings ++
Seq( name := "foo")
) aggregate(core, cli, web)
// mainClass:= Some("Main"))
lazy val core : Project = Project("core", file("core"), delegates = root :: Nil, settings = buildSettings ++
Seq(
name := "foo-core",
libraryDependencies ++= logDeps ++ Seq(scalaTest, apacheHttpClient, apacheHttpCore)
)
)
lazy val cli: Project = Project("cli", file("cli"), settings = buildSettings ++
Seq(
name := "foo-cli",
libraryDependencies ++= logDeps ++ Seq(apacheHttpClient),
fork in run := true,
javaOptions in run += "-Djava.library.path=/home/jolivier/Projets/asknow/lib/jnotify-lib-0.93"
)) dependsOn(core) settings(
)
lazy val web: Project = Project("web", file("web"), settings = buildSettings ++
Seq (resolvers := Seq(webPluginRepo, jettyRepo),
name := "foo-http",
libraryDependencies ++= logDeps ++ webPluginDeps ++ liftDeps
) ++
webSettings
) dependsOn(core)
}
When I try sbt jetty-run I get the following error message:
[error] Not a valid command: jetty-run
[error] Not a valid project ID: jetty-run
[error] Not a valid configuration: jetty-run
[error] Not a valid key: jetty-run (similar: jetty-port, jetty-context, run)
[error] jetty-run
[error]
So I noticed that some jetty-* commands do exist, but not the one I want, so I printed webSettings which is supposed to contain all these new settings and it contains jetty-context and jetty-port, as well as jetty-configuration and others, but not jetty-run :s.
What did I go wrong to not have jetty-run?
I tried switching to scala-2.8.1 since the web plugin is currently only on scala 2.8.1, by changing my buildScalaVersion variable but that didn't change anything. Do you have any idea?
Thanks in advance for your help
Tasks are aggregated; commands are not.
jetty-run is a command. It is only available in the context of the sub-project with the web plugin settings.
> project web
> jetty-run
Once it is running, you can use the prepare-webapp task to redeploy the webapp. This can be run from the context of the root project, because it aggregates the web project.