I have various library dependencies in my build.sbt file
scalaVersion := "2.10.5",
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "2.2.5" % "test",
"org.apache.spark" %% "spark-core" % "1.6.1" % "provided",
"commons-logging" % "commons-logging" % "1.2"
) map (_.excludeAll(
ExclusionRule(organization = "org.slf4j"),
ExclusionRule(organization = "log4j"),
ExclusionRule(organization = "javax.servlet")
)),
libraryDependencies ++= Seq(("org.slf4j" % "slf4j-log4j12" % "1.7.10")
.excludeAll(ExclusionRule(organization = "log4j"))),
libraryDependencies += "log4j" % "log4j" % "1.2.16" % "test",
I can see them in my external libraries in the project view.
However, the classes in the libraries are not being picked up in the Scala source files.
This is a sub module.
And the dependencies are declared in the parent, build.sbt. The code is in the child project.
I have tried:
Invalidating Caches / Restarts...
Deleting the .idea file and re-importing
Building project several times
Any help appreciated.
The dependencies needs to be explicitly added also in the subproject.
For example, in your build.sbt in the root folder of your project you can set:
val commonDependencies = Seq(
"commons-io" % "commons-io" % "2.5"
// other dependencies
)
val root = (project in file(".")).settings(
libraryDependencies ++= commonDependencies,
libraryDependencies ++= Seq(
// dependencies only for project root
)
)
val topWordsCounter = (project in file("top-words-counter"))
.settings(
libraryDependencies ++= commonDependencies,
libraryDependencies ++= Seq(
// dependencies only for the sub project project top-words-counter
)
)
Hope this helps
Related
I am trying to upgrade mumoshu's Memcached Plugin for Play framework, to Play 2.8.6, in build.sbt.
The upgrade instructions are:
For Play 2.6.x and newer: !!! Changed play.modules.cache.* config keys to play.cache.* !!!
val appDependencies = Seq(
play.PlayImport.cacheApi,
"com.github.mumoshu" %% "play2-memcached-play26" % "0.9.2"
)
val main = Project(appName).enablePlugins(play.PlayScala).settings(
version := appVersion,
libraryDependencies ++= appDependencies,
resolvers += "Spy Repository" at "http://files.couchbase.com/maven2" // required to resolve `spymemcached`, the plugin's dependency.
)
Before the upgrade, I had:
libraryDependencies ++= Seq(
jdbc,
cache,
ws,
"com.github.mumoshu" %% "play2-memcached-play24" % "0.7.0",
//...
}
After the (unsuccessful) upgrade:
the cache dependency is not resolved, so I removed it.
The updated build.sbt now looks like this:
name := "CH07"
version := "1.0"
lazy val `ch07` = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.13.4"
resolvers += "Spy Repository" at "http://files.couchbase.com/maven2"
val appDependencies = Seq(
play.sbt.PlayImport.cache,
"com.github.mumoshu" %% "play2-memcached-play28" % "0.11.0"
//"com.github.mumoshu" %% "play2-memcached-play26" % "0.9.2"
)
val main = (project in file(".")).enablePlugins(play.sbt.PlayScala).settings(
version := version.value,
libraryDependencies ++= appDependencies,
resolvers += "Spy Repository" at "http://files.couchbase.com/maven2" // required to resolve `spymemcached`, the plugin's dependency.
)
libraryDependencies ++= Seq(
jdbc,
//cache,
ws,
"org.hsqldb" % "hsqldb" % "2.5.0",
"org.jooq" % "jooq" % "3.14.4",
"org.jooq" % "jooq-codegen-maven" % "3.14.4",
"org.jooq" % "jooq-meta" % "3.14.4",
"joda-time" % "joda-time" % "2.7",
"com.github.ironfish" %% "akka-persistence-mongo-casbah" % "0.7.6",
"com.ning" % "async-http-client" % "1.9.29"
)
routesGenerator := InjectedRoutesGenerator
Errors I am getting:
* build.sbt:
error: value cache is not a member of object play.sbt.PlayImport
* build.sbt:
"com.github.mumoshu" %% "play2-memcached-play28" % "0.11.0"
Intellij says: unresolved artifact. Not resolved or indexed.
* in project/plugins.sbt:
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.8.6")
Intellij says: unresolved artifact. Not resolved or indexed.
Can you see any errors I made in the build.sbt?
In other words, I am looking for a working build.sbt (and the other required definitions) example for a project that uses Play 2.8.6 and the Memcached Plugin for Play framework 0.9.2. (or highest possible version of the latter).
UPDATE:
This is my current configuration which does compile:
build.sbt
name := "CH07"
version := "1.0"
lazy val `ch07` = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.13.4"
resolvers += "Spy Repository" at "http://files.couchbase.com/maven2"
libraryDependencies ++= Seq(
jdbc,
cacheApi,
ws,
"com.github.mumoshu" %% "play2-memcached-play28" % "0.11.0",
"org.hsqldb" % "hsqldb" % "2.5.0",
"org.jooq" % "jooq" % "3.14.4",
"org.jooq" % "jooq-codegen-maven" % "3.14.4",
"org.jooq" % "jooq-meta" % "3.14.4",
"joda-time" % "joda-time" % "2.7",
"com.ning" % "async-http-client" % "1.9.29",
"com.github.scullxbones" %% "akka-persistence-mongo-common" % "3.0.5",
"com.typesafe.akka" %% "akka-persistence" % "2.6.10",
"com.typesafe.akka" %% "akka-persistence-query" % "2.6.10",
"com.typesafe.akka" %% "akka-persistence-typed" % "2.6.10",
"com.typesafe.play" %% "play-guice" % "2.8.7"
)
routesGenerator := InjectedRoutesGenerator
project/build.properties
sbt.version=1.3.10
project/plugins.sbt
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.8.7")
How about this?
name := """pla-memcache-example"""
organization := "com.example"
version := "1.0-SNAPSHOT"
scalaVersion := "2.13.3"
val appDependencies = Seq(
guice,
play.PlayImport.cacheApi,
"com.github.mumoshu" %% "play2-memcached-play28" % "0.11.0",
"org.scalatestplus.play" %% "scalatestplus-play" % "5.0.0" % Test
)
lazy val root = (project in file(".")).enablePlugins(PlayScala).settings(
version := appVersion,
libraryDependencies ++= appDependencies,
resolvers += "Spy Repository" at "http://files.couchbase.com/maven2" // required to resolve `spymemcached`, the plugin's dependency.
)
plugins.sbt
sbt.version=1.3.13
plugins.sbt
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.8.7")
In this case, first find the giter8 template and use the "sbt new" command.
Next, go to the official page of the plugin and check the name of the library, and then use Maven search to find the version you want to use.
It's a good idea to read the entire readme of the plugin without skipping it.
You can also try to run sbt with commands instead of using IntelliJ.
If that doesn't solve the problem, you can try clearing the cache in the .ivy2 directory.
Reference Links:
https://github.com/mumoshu/play2-memcached
https://github.com/playframework/play-scala-seed.g8
https://search.maven.org/artifact/com.github.mumoshu/play2-memcached-play28_2.13/0.11.0/jar
Your issue is when trying to import play.sbt.PlayImport.cache. According to Cache APIs Migration by play:
New packages
Now `cache` has been split into a `cacheApi` component with just the API, and `ehcache` that contains the Ehcache implementation. If you are using the default Ehcache implementation, simply change `cache` to `ehcache` in your `build.sbt`:
libraryDependencies ++= Seq(
ehcache
)
If you are defining a custom cache API, or are writing a cache implementation module, you can just depend on the API:
libraryDependencies ++= Seq(
cacheApi
)
Removed APIs
The deprecated Java class play.cache.Cache was removed and you now must inject an play.cache.SyncCacheApi or play.cache.AsyncCacheApi.
Therefore a complete build.sbt will be:
name := "CH07"
version := "1.0"
lazy val `ch07` = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.13.4"
resolvers += "Spy Repository" at "https://files.couchbase.com/maven2"
val appDependencies = Seq(
play.sbt.PlayImport.ehcache,
play.sbt.PlayImport.cacheApi,
"com.github.mumoshu" %% "play2-memcached-play28" % "0.11.0"
//"com.github.mumoshu" %% "play2-memcached-play26" % "0.9.2"
)
val main = (project in file(".")).enablePlugins(play.sbt.PlayScala).settings(
version := version.value,
libraryDependencies ++= appDependencies,
resolvers += "Spy Repository" at "https://files.couchbase.com/maven2" // required to resolve `spymemcached`, the plugin's dependency.
)
libraryDependencies ++= Seq(
jdbc,
//cache,
ws,
"org.hsqldb" % "hsqldb" % "2.5.0",
"org.jooq" % "jooq" % "3.14.4",
"org.jooq" % "jooq-codegen-maven" % "3.14.4",
"org.jooq" % "jooq-meta" % "3.14.4",
"joda-time" % "joda-time" % "2.7",
"com.github.scullxbones" %% "akka-persistence-mongo-common" % "3.0.5",
"com.ning" % "async-http-client" % "1.9.29"
)
routesGenerator := InjectedRoutesGenerator
build.properties:
sbt.version=1.4.4
plugins.sbt:
logLevel := Level.Warn
resolvers += "Typesafe repository" at "https://repo.typesafe.com/typesafe/releases/"
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.8.7")
I am working on a big SBT project with many sub-projects (SBT ver. 0.13.17).
In the build.sbt of the subprojects, the dependencies are added to libraryDependencies:
lazy val root = Project("subproject-name", file("."))
.dependsOn(someOtherSubproject1)
.dependsOn(someOtherSubproject2)
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.5" % "test",
"org.json4s" %% "json4s-native" % "3.5.3"
)
However, in many examples on the web I see that the dependencies are set as part of the 'settings' of the project:
lazy val root = Project("subproject-name", file("."))
.dependsOn(someOtherSubproject1)
.dependsOn(someOtherSubproject2)
.settings(
subprojectSettings,
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.5" % "test",
"org.json4s" %% "json4s-native" % "3.5.3"
)
)
Now, this structural difference is relevant also to other settings keys (e.g. parallelExecution, scalacOptions, updateOptions etc.).
My question: what is the internal differences between the two structures? How do they affect the workings of the SBT?
I am building a SBT multi-project project, which has common module and logic module, and logic.dependsOn(common).
In common, SparkSQL 2.2.1 ("org.apache.spark" %% "spark-sql" % "2.2.1") is introduced. In logic, SparkSQL also is used, however I get compilation errors, saying "object spark is not a member of package org.apache".
Now, if I add SparkSQL dependency to logic as "org.apache.spark" %% "spark-sql" % "2.2.1", it works. However if I add "org.apache.spark" %% "spark-sql" % "2.2.1" % Provided", I get the same error.
I don't get why this happened, why not the dependency can't be transitive from common to logic
here is the root sbt files:
lazy val commonSettings = Seq(
organization := "...",
version := "0.1.0",
scalaVersion := "2.11.12",
resolvers ++= Seq(
clojars,
maven_local,
novus,
twitter,
spark_packages,
artima
),
test in assembly := {},
assemblyMergeStrategy in assembly := {...}
)
lazy val root = (project in file(".")).aggregate(common, logic)
lazy val common = (project in file("common")).settings(commonSettings:_*)
lazy val logic = (project in file("logic")).dependsOn(common).settings(commonSettings:_*)
here is the logic module sbt file:
libraryDependencies ++= Seq(
spark_sql.exclude("io.netty", "netty"),
embedded_elasticsearch % "test",
scalatest % "test"
)
dependencyOverrides ++= Seq(
"com.fasterxml.jackson.core" % "jackson-core" % "2.6.5",
"com.fasterxml.jackson.core" % "jackson-databind" % "2.6.5",
"com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.6.5",
"com.fasterxml.jackson.core" % "jackson-annotation" % "2.6.5",
"org.json4s" %% "json4s-jackson" % "3.2.11"
)
assemblyJarName in assembly := "***.jar"
I am trying to create a Play application with a back end and a front end(using Scala.js). I separated the code into client and server folders.
For creating a Multi Project Build I looked at this http://www.scala-sbt.org/release/docs/Multi-Project.html and make my build.sbt like this:
name := """ScalaWeb"""
version := "1.0-SNAPSHOT"
lazy val commonSettings = Seq(
scalaVersion := "2.11.7",
libraryDependencies ++= Seq(
jdbc,
cache,
ws,
specs2 % Test
),
libraryDependencies ++= Seq(
"org.sorm-framework" % "sorm" % "0.3.19",
"org.scala-lang" % "scala-compiler" % scalaVersion.value force(),
"com.h2database" % "h2" % "1.3.148"
),
libraryDependencies += "com.typesafe.scala-logging" %% "scala-logging" % "3.1.0",
/*libraryDependencies += "com.typesafe.scala-logging" %% "scala-logging" % "3.1.0",
libraryDependencies += "org.slf4j" % "slf4j-api" % "1.7.12"
libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.1.3"*/
resolvers += "scalaz-bintray" at "http://dl.bintray.com/scalaz/releases",
// Play provides two styles of routers, one expects its actions to be injected, the
// other, legacy style, accesses its actions statically.
routesGenerator := InjectedRoutesGenerator,
scalacOptions += "-Ylog-classpath",
// use this if you just want jawn's parser, and will implement your own facade
libraryDependencies += "org.spire-math" %% "jawn-parser" % "0.8.3",
// use this if you want jawn's parser and also jawn's ast
libraryDependencies += "org.spire-math" %% "jawn-ast" % "0.8.3",
//for reading Json libraries
libraryDependencies += "org.scalaj" %% "scalaj-http" % "2.2.1"
)
lazy val server = (project in file("server")).settings(
commonSettings: _*
).enablePlugins(PlayScala)
lazy val client = (project in file("client")).settings(
commonSettings: _*
).enablePlugins(PlayScala)
fork in run := true
So basically I placed all dependencies into commonSettings and the imported that into the client and server projects.
However, I am getting this error:
[error] (scalaweb/compile:backgroundRun) No main class detected.
Why is this occuring?
Have I set up my build.sbt incorrectly?
That's because your root project (scalaweb) doesn't have a main class.
You can either:
run server's run command by doing: server/run
aggregate client and server under root project
(http://www.scala-sbt.org/0.13/docs/Multi-Project.html#Aggregation)
This might do what you wish.
I am using sbt to build a multi module project, and I want to use sbt-buildinfo to allow one of the services to have this info in it for debugging and versioning purposes.
The problem with this, is that the sbt-buildinfo plugin seems to only take the library dependencies of the root project and not all the dependencies of the aggregated sub projects. It seems to me that if you run the plugin after the aggregation then the root project would have those dependencies in it and would show up, but they do not. Maybe I'm just not understanding the process here, but that seems reasonable to me.
Now, this worked perfectly fine with one project, but with many sub projects it no longer works. In addition now it no longer even generates the File, I don't know what I'm doing wrong at this point. Here is my build. Any help is appreciated.
EclipseKeys.createSrc := EclipseCreateSrc.Default + EclipseCreateSrc.Managed
lazy val sharedSettings = Seq(
organization := "com.planalytics",
version := "0.0.1-SNAPSHOT",
name := "Ingestions",
scalaVersion := "2.11.7",
exportJars := true
) ++ net.virtualvoid.sbt.graph.Plugin.graphSettings ++ Defaults.defaultSettings
lazy val sparkVersion = "1.5.1"
lazy val akkaVersion = "2.3.12"
lazy val sprayVersion = "1.3.3"
lazy val hadoopVersion = "2.7.1"
lazy val exclusionRules = Seq(ExclusionRule(organization = "org.slf4j"),
ExclusionRule(organization = "org.apache.hadoop"))
lazy val sharedDeps = Seq(libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-log4j12" % "1.7.10",
"org.scalacheck" %% "scalacheck" % "1.12.0" % "test",
"org.scalatest" %% "scalatest" % "2.2.4" % "test",
"org.scalanlp" %% "breeze-natives" % "0.11.2"))
lazy val sparkDeps = Seq(libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion excludeAll(exclusionRules: _*),
"org.apache.spark" %% "spark-sql" % sparkVersion excludeAll(exclusionRules: _*),
"org.apache.spark" %% "spark-mllib" % sparkVersion excludeAll(exclusionRules: _*),
"com.databricks" %% "spark-csv" % "1.2.0" excludeAll(exclusionRules: _*),
"org.apache.hadoop" % "hadoop-common" % hadoopVersion,
"org.apache.hadoop" % "hadoop-client" % hadoopVersion,
"org.apache.hadoop" % "hadoop-hdfs" % hadoopVersion,
"org.apache.hadoop" % "hadoop-aws" % hadoopVersion))
lazy val akkaDeps = Seq(libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-cluster" % akkaVersion,
"com.typesafe.akka" %% "akka-remote" % akkaVersion,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion))
lazy val sprayDeps = Seq(libraryDependencies ++= Seq(
"io.spray" %% "spray-can" % sprayVersion,
"io.spray" %% "spray-routing" % sprayVersion,
"io.spray" %% "spray-json" % "1.3.2"))
lazy val root = (project in file(".")).
settings(sharedSettings: _*).
settings(
scalacOptions += "-deprecation",
javacOptions ++= Seq("-source", "1.8", "target", "1.8"),
javaOptions += "-Xmx2G").aggregate(ingestion, services, util).enablePlugins(BuildInfoPlugin).
settings(
buildInfoKeys := Seq[BuildInfoKey](name, version, scalaVersion, sbtVersion),
buildInfoOptions += BuildInfoOption.BuildTime,
buildInfoPackage := "com.planalytics",
buildInfoObject := "BuildInfo"
)
lazy val ingestion = Project(
id = "Ingestion-Engine",
base = file("ingestion"),
settings = Project.defaultSettings ++ sharedSettings ++ sparkDeps ++ akkaDeps ++ sharedDeps).
settings(
name := "Ingestion-Engine",
libraryDependencies ++= Seq(
"com.github.seratch" %% "awscala" % "0.5.+",
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % "test"
)
).dependsOn(util)
lazy val services = Project(
id = "Ingestion-Services",
base = file("services"),
settings = Project.defaultSettings ++ sharedSettings ++ sprayDeps ++ sharedDeps).
settings(
name := "Ingestion-Services",
libraryDependencies ++= Seq(
"io.spray" %% "spray-testkit" % sprayVersion % "test"
)
).dependsOn(ingestion)
lazy val util = Project(
id = "Ingestion-Util",
base = file("util"),
settings = Project.defaultSettings ++ sharedSettings ++ sparkDeps ++ sharedDeps).
settings(
name := "Ingestion-Utils",
libraryDependencies ++= Seq(
"net.ceedubs" %% "ficus" % "1.1.2"
),
dependencyOverrides ++= Set(
"com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)
)
I'm also open to any thoughts on my build in general, I feel like I'm doing some things wrong or repetitively, so any suggestions there is appreciated as well. Thanks!