Why subprojects does not recognize dependencies? - scala

I've defined two sub projects that looks as follow:
val Http4sVersion = "0.21.0-M4"
val CirceVersion = "0.12.1"
val Specs2Version = "4.7.0"
val LogbackVersion = "1.2.3"
val ScalaTestVersion = "3.0.8"
val TestContainerVersion = "1.11.3"
val KafkaTestContainerVersion = "1.11.3"
val ConfigVersion = "1.3.4"
val SpringVersion = "5.1.8.RELEASE"
val CatsVersion = "2.0.0"
lazy val settings = Seq(
organization := "com.sweetsoft",
name := "connector",
scalaVersion := "2.13.0",
addCompilerPlugin("org.typelevel" %% "kind-projector" % "0.10.3"),
addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.3.0"),
scalacOptions ++= Seq(
"-deprecation",
"-encoding", "UTF-8",
"-language:higherKinds",
"-language:postfixOps",
"-feature",
"-Xfatal-warnings",
),
scalacOptions in(Compile, console) ~= {
_.filterNot(Set("-Xlint"))
}
)
lazy val dependencies = Seq(
"org.http4s" %% "http4s-blaze-server" % Http4sVersion,
"org.http4s" %% "http4s-blaze-client" % Http4sVersion,
"org.http4s" %% "http4s-circe" % Http4sVersion,
"org.http4s" %% "http4s-dsl" % Http4sVersion,
"io.circe" %% "circe-generic" % CirceVersion,
"ch.qos.logback" % "logback-classic" % LogbackVersion,
"org.typelevel" %% "cats-core" % CatsVersion,
"com.typesafe" % "config" % ConfigVersion % "test",
"org.scalactic" %% "scalactic" % ScalaTestVersion % "test",
"org.scalatest" %% "scalatest" % ScalaTestVersion % "test",
"org.testcontainers" % "testcontainers" % TestContainerVersion % "test",
"org.testcontainers" % "kafka" % KafkaTestContainerVersion % "test",
"org.springframework" % "spring-core" % SpringVersion % "test",
"org.typelevel" %% "cats-laws" % CatsVersion % "test",
"com.github.alexarchambault" %% "scalacheck-shapeless_1.14" % "1.2.3" % "test",
"org.scalacheck" %% "scalacheck" % "1.14.0" % "test"
)
lazy val global = project
.in(file("."))
.settings(
settings,
libraryDependencies ++= dependencies
)
.aggregate(core, serversupervisor)
lazy val core = (project in file("core"))
.settings(settings)
lazy val serversupervisor = (project in file("serversupervisor"))
.settings(settings)
.dependsOn(core)
As you can see, the two subprojects are core and serversupervisor.
The problem is, that those two subprojects does not recognize dependencies:
I am using Intellj and as you can see, it does not recognize the dependencies.
What am I doing wrong?

Put libraryDependencies ++= dependencies into settings.
global, core and serversupervisor are three different subprojects. They can have different library dependencies. Currently you add them to global but not to core and serversupervisor.
Alternatively you can move libraryDependencies ++= dependencies to Global or
ThisBuild scope rather than specific subproject scope. You can add at top
ThisBuild / libraryDependencies ++= dependencies
or even
Global / libraryDependencies ++= dependencies
https://www.scala-sbt.org/1.x/docs/Multi-Project.html
https://www.scala-sbt.org/1.x/docs/Scopes.html

Related

"datasource not a member of org.apache.phoenix" when trying to Save DataFrames to Phoenix using DataSourceV2

I am trying to Save DataFrames to Phoenix using DataSourceV2 following the below mentioned source:
Apache Spark plugin
I created a dataframe and I want to save it to phoenix in the following way:
import org.apache.spark.SparkContext
import org.apache.phoenix.spark.datasource.v2.PhoenixDataSource
val conf = new SparkConf().setAppName("Spark sql to convert rdd to df")
val sc = new SparkContext(conf)
val sqlContext= new org.apache.spark.sql.SQLContext(sc)
import sqlContext.implicits._
val MasterDF = MasterRecordSeq.toDF()
MasterDF.write
.format("phoenix")
.mode(SaveMode.Overwrite)
.options(Map("table" -> masterTableName, PhoenixDataSource.ZOOKEEPER_URL -> "phoenix-server:2181"))
.save()
But the import org.apache.phoenix.spark.datasource.v2.PhoenixDataSource is not being recognized. It throws the following error:
object datasource is not a member of package org.apache.phoenix.spark
I have searched through a lot of internet but I'm not able to find what the bug is.
The following are the dependencies I added in build.sbt:
libraryDependencies += "org.apache.phoenix" % "phoenix-spark" % "5.0.0-HBase-2.0"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.4.5"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.5"
libraryDependencies += "org.apache.phoenix" % "phoenix-core" % "5.0.0-HBase-2.0"
The following is the complete build file:
import NativePackagerHelper._
import java.util.Properties
import com.typesafe.sbt.packager.MappingsHelper._
//import sbtrelease.ReleaseStateTransformations._
name := """gavel"""
//scapegoatVersion in ThisBuild := "1.1.0"
//version := sys.env.get("BUILD_NUMBER").getOrElse("3.0-LOCAL")
version := "3.0"
scalaVersion := "2.11.12"
//crossScalaVersions := Seq("2.11.11", "2.12.3")
//scapegoatVersion in ThisBuild := "1.3.5"
scalaBinaryVersion in ThisBuild := "2.12"
javacOptions ++= Seq("-source", "1.6", "-target", "1.6")
scalacOptions ++= Seq("-unchecked", "-deprecation", "-feature")
scalacOptions in (Compile, doc) ++= Seq("-unchecked", "-deprecation", "-diagrams", "-implicits", "-skip-packages", "samples")
lazy val root = (project in file(".")).enablePlugins(PlayScala,sbtdocker.DockerPlugin,JavaAppPackaging).settings(
watchSources ++= (baseDirectory.value / "public/frontend" ** "*").get
)
mainClass := Some("play.core.server.ProdServerStart")
fullClasspath in assembly += Attributed.blank(PlayKeys.playPackageAssets.value)
mappings in Universal ++= directory(baseDirectory.value / "public")
unmanagedBase := baseDirectory.value / "libs"
routesGenerator := InjectedRoutesGenerator
resolvers += "scalaz-bintray" at "https://dl.bintray.com/scalaz/releases"
libraryDependencies ++= Seq(
"com.typesafe" % "config" % "1.3.1",
"mysql" % "mysql-connector-java" % "5.1.34",
"com.typesafe.play" %% "play-slick" % "3.0.0",
"com.typesafe.play" %% "play-slick-evolutions" % "3.0.0",
"com.typesafe.play" %% "play-json" % "2.6.0",
"org.scalatestplus.play" %% "scalatestplus-play" % "3.0.0" % "test",
specs2 % Test,
// "io.rest-assured" % "rest-assured" % "3.0.0" % "test",
// "io.rest-assured" % "scala-support" % "3.0.0" % "test",
// "com.squareup.okhttp" % "mockwebserver" % "2.5.0" % "test",
"javax.mail" % "mail" % "1.4",
"io.swagger" %% "swagger-play2" % "1.6.1",
"com.fasterxml.jackson.core" % "jackson-databind" % "2.4.0",
"com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.4.0",
"com.google.code.gson" % "gson" % "1.7.1",
"commons-io" % "commons-io" % "2.4",
"com.typesafe.akka" %% "akka-actor" % "2.4.16",
"com.typesafe.akka" %% "akka-testkit" % "2.4.16" % "test",
"org.typelevel" %% "macro-compat" % "1.1.1",
"org.scala-lang" % "scala-reflect" % scalaVersion.value % "provided",
"org.scalatest" %% "scalatest" % "3.0.0" % "test",
compilerPlugin("org.scalamacros" %% "paradise" % "2.1.0" cross CrossVersion.full),
guice
)
libraryDependencies ++= Seq(
"com.101tec" % "zkclient" % "0.4",
"org.apache.kafka" % "kafka_2.10" % "0.8.1.1"
exclude("javax.jms", "jms")
exclude("com.sun.jdmk", "jmxtools")
exclude("com.sun.jmx", "jmxri")
)
libraryDependencies += ws
libraryDependencies += ehcache
// https://mvnrepository.com/artifact/org.apache.phoenix/phoenix-spark
libraryDependencies += "org.apache.phoenix" % "phoenix-spark" % "5.0.0-HBase-2.0"
libraryDependencies += "com.google.protobuf" % "protobuf-java" % "2.4.0"
libraryDependencies += "org.codehaus.jackson" % "jackson-mapper-asl" % "1.9.13"
libraryDependencies += "com.google.code.gson" % "gson" % "2.3"
libraryDependencies += "org.apache.phoenix" % "phoenix-queryserver-client" % "4.13.1-HBase-1.2"
libraryDependencies += "com.github.takezoe" %% "solr-scala-client" % "0.0.19"
libraryDependencies += "com.squareup.okhttp" % "okhttp" % "2.7.0"
libraryDependencies += "org.threeten" % "threetenbp" % "1.2"
libraryDependencies += "io.gsonfire" % "gson-fire" % "1.0.1"
libraryDependencies += "au.com.bytecode" % "opencsv" % "2.4"
libraryDependencies += "org.simplejavamail" % "simple-java-mail" % "5.0.8"
libraryDependencies += "org.apache.solr" % "solr-solrj" % "6.6.2"
libraryDependencies += "com.jcraft" % "jsch" % "0.1.55"
libraryDependencies += "com.vmware" % "vijava" % "5.1"
libraryDependencies += "com.microsoft.sqlserver" % "mssql-jdbc" % "6.1.0.jre8" % Test
//libraryDependencies += "com.microsoft.sqlserver" % "sqljdbc4" % "4.0"
libraryDependencies += "org.apache.poi" % "poi" % "3.17"
libraryDependencies += "org.apache.poi" % "poi-ooxml" % "3.17"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.4.5"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.5"
libraryDependencies += "org.apache.phoenix" % "phoenix-core" % "5.0.0-HBase-2.0"
crossSbtVersions := Seq("0.13.17", "1.1.6")
publishTo := {
val isSnapshotValue = isSnapshot.value
val nexus = "https://oss.sonatype.org/"
if(isSnapshotValue) Some("snapshots" at nexus + "content/repositories/snapshots")
else Some("releases" at nexus + "service/local/staging/deploy/maven2")
}
publishMavenStyle := true
publishArtifact in Test := false
parallelExecution in Test := false
dockerfile in docker := {
// The assembly task generates a fat JAR file
val artifact: File = assembly.value
val artifactTargetPath = s"/app/${artifact.name}"
new Dockerfile {
from("java")
from("mysql:5.7")
add(artifact, artifactTargetPath)
entryPoint("java", "-jar", artifactTargetPath)
}
}
val appProperties = settingKey[Properties]("The application properties")
appProperties := {
val prop = new Properties()
IO.load(prop, new File("./conf/database.conf"))
prop
}
javaOptions in Test += "-Dconfig.file=conf/application.test.conf"
resolvers += "Sonatype snapshots" at "http://oss.sonatype.org/content/repositories/snapshots/"
sourceDirectories in (Compile, TwirlKeys.compileTemplates) :=
(unmanagedSourceDirectories in Compile).value
flywayDriver := "com.mysql.jdbc.Driver"
flywayUrl := appProperties.value.getProperty("slick.dbs.default.db.url").replaceAll("\"", "")
flywayUser := appProperties.value.getProperty("slick.dbs.default.db.user")
flywayPassword := appProperties.value.getProperty("slick.dbs.default.db.password").replaceAll("\"", "")
flywayLocations := Seq("filesystem:conf/db/default")
fork in run := true
//coverageEnabled := false
//coverageMinimum := 70
//coverageFailOnMinimum := true
//coverageHighlighting := true
publishArtifact in Test := false
parallelExecution in Test := false
enablePlugins(SbtProguard)
import com.lightbend.sbt.SbtProguard._
javaOptions in (Proguard, proguard) := Seq("-Xmx2G")
proguardOptions in Proguard ++= Seq("-dontnote", "-dontwarn", "-ignorewarnings")
proguardOptions in Proguard += ProguardOptions.keepMain("some.MainClass")
proguardMergeStrategies in Proguard += ProguardMerge.append("*.conf")
proguardMergeStrategies in Proguard ++= Seq(
ProguardMerge.discard("\\.zip$".r),
ProguardMerge.discard("\\.xml$".r),
ProguardMerge.discard("\\.txt$".r),
ProguardMerge.discard("\\.conf$".r),
ProguardMerge.discard("\\.jar$".r)
)
My phoenix version is 5.0. My Hbase version is 2.0.2.3.1.0.0-78. Am I missing any configuration?
I had the same problem (error), but in my specific case it was for a scala script in a Hortonworks Big Data cluster to be executed by Spark
I managed to solve it by compiling the phoenix-spark repository available on github and importing the jar into the spark directory.
Here are the commands I ran to build the jar, I hope it helps.
sudo yum install maven
wget https://github.com/apache/phoenix-connectors/archive/master.zip
unzip master.zip
cd phoenix-connectors/phoenix-spark
mvn clean compile
mvn package
cd target/scala-2.12/
cp phoenix-spark-1.0.0-SNAPSHOT.jar /usr/hdp/current/spark2-client/jars

`java.lang.NoSuchMethodError: cats.FlatMap.map2` in runtime when using `.sequence`

I'm getting the following runtime error after migrating from cats v1.1.0 to v1.4.0 (An error arises from places where .sequence (cats.Traverse) is used).
The code looks like:
import cats.implicits._
import cats.effect.IO
List(1, 2, 3).map(x => IO(...)).sequence
java.lang.NoSuchMethodError: cats.FlatMap.map2$(Lcats/FlatMap;Ljava/lang/Object;Ljava/lang/Object;Lscala/Function2;)Ljava/lang/Object;
at cats.effect.IOLowPriorityInstances$IOEffect.map2(IO.scala:765)
...
Here is my build.sbt:
organization := "org.xxx"
name := "yyy"
version := "0.0.1"
scalaVersion := "2.12.9"
resolvers += Resolver.bintrayRepo("hseeberger", "maven")
resolvers ++= Seq("Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/")
lazy val doobieVersion = "0.5.3"
lazy val akkaHttpVersion = "10.1.1"
lazy val akkaVersion = "2.5.12"
lazy val catsVersion = "1.4.0"
lazy val circeVersion = "0.9.3"
lazy val doobieDeps = Seq(
"org.tpolecat" %% "doobie-core" % doobieVersion,
"org.tpolecat" %% "doobie-postgres" % doobieVersion,
"org.tpolecat" %% "doobie-scalatest" % doobieVersion,
"org.tpolecat" %% "doobie-hikari" % doobieVersion
)
lazy val catsDeps = Seq(
"org.typelevel" %% "cats-effect" % catsVersion,
"org.typelevel" %% "cats-core" % catsVersion
)
lazy val otherDeps = Seq(
"com.github.pureconfig" %% "pureconfig" % "0.9.1",
"org.scorexfoundation" %% "scrypto" % "2.1.1",
"de.heikoseeberger" %% "akka-http-circe" % "1.20.1",
"org.scalaj" %% "scalaj-http" % "2.4.0",
"org.flywaydb" % "flyway-core" % "5.1.1",
"com.github.blemale" %% "scaffeine" % "2.5.0",
("org.scorexfoundation" %% "sigma-state" % "master-2b4b07a1-SNAPSHOT")
.exclude("ch.qos.logback", "logback-classic")
.exclude("org.scorexfoundation", "scrypto"),
)
lazy val circeDeps = Seq(
"io.circe" %% "circe-core" % circeVersion,
"io.circe" %% "circe-parser" % circeVersion,
"io.circe" %% "circe-generic" % circeVersion
)
libraryDependencies ++= (otherDeps ++ doobieDeps ++ catsDeps ++ loggingDeps ++ akkaDeps ++ circeDeps ++ testDeps)
I've tried to run it different ways (idea, set run) and on different platforms - the result is always the same.
What could it be caused by?
Try to change versions to
"org.typelevel" %% "cats-effect" % "1.4.0",
"org.typelevel" %% "cats-core" % "1.6.1"
Your project seems to work with them.

sbtbuildinfo - Building with multiple projects

I am using sbt to build a multi module project, and I want to use sbt-buildinfo to allow one of the services to have this info in it for debugging and versioning purposes.
The problem with this, is that the sbt-buildinfo plugin seems to only take the library dependencies of the root project and not all the dependencies of the aggregated sub projects. It seems to me that if you run the plugin after the aggregation then the root project would have those dependencies in it and would show up, but they do not. Maybe I'm just not understanding the process here, but that seems reasonable to me.
Now, this worked perfectly fine with one project, but with many sub projects it no longer works. In addition now it no longer even generates the File, I don't know what I'm doing wrong at this point. Here is my build. Any help is appreciated.
EclipseKeys.createSrc := EclipseCreateSrc.Default + EclipseCreateSrc.Managed
lazy val sharedSettings = Seq(
organization := "com.planalytics",
version := "0.0.1-SNAPSHOT",
name := "Ingestions",
scalaVersion := "2.11.7",
exportJars := true
) ++ net.virtualvoid.sbt.graph.Plugin.graphSettings ++ Defaults.defaultSettings
lazy val sparkVersion = "1.5.1"
lazy val akkaVersion = "2.3.12"
lazy val sprayVersion = "1.3.3"
lazy val hadoopVersion = "2.7.1"
lazy val exclusionRules = Seq(ExclusionRule(organization = "org.slf4j"),
ExclusionRule(organization = "org.apache.hadoop"))
lazy val sharedDeps = Seq(libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-log4j12" % "1.7.10",
"org.scalacheck" %% "scalacheck" % "1.12.0" % "test",
"org.scalatest" %% "scalatest" % "2.2.4" % "test",
"org.scalanlp" %% "breeze-natives" % "0.11.2"))
lazy val sparkDeps = Seq(libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion excludeAll(exclusionRules: _*),
"org.apache.spark" %% "spark-sql" % sparkVersion excludeAll(exclusionRules: _*),
"org.apache.spark" %% "spark-mllib" % sparkVersion excludeAll(exclusionRules: _*),
"com.databricks" %% "spark-csv" % "1.2.0" excludeAll(exclusionRules: _*),
"org.apache.hadoop" % "hadoop-common" % hadoopVersion,
"org.apache.hadoop" % "hadoop-client" % hadoopVersion,
"org.apache.hadoop" % "hadoop-hdfs" % hadoopVersion,
"org.apache.hadoop" % "hadoop-aws" % hadoopVersion))
lazy val akkaDeps = Seq(libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-cluster" % akkaVersion,
"com.typesafe.akka" %% "akka-remote" % akkaVersion,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion))
lazy val sprayDeps = Seq(libraryDependencies ++= Seq(
"io.spray" %% "spray-can" % sprayVersion,
"io.spray" %% "spray-routing" % sprayVersion,
"io.spray" %% "spray-json" % "1.3.2"))
lazy val root = (project in file(".")).
settings(sharedSettings: _*).
settings(
scalacOptions += "-deprecation",
javacOptions ++= Seq("-source", "1.8", "target", "1.8"),
javaOptions += "-Xmx2G").aggregate(ingestion, services, util).enablePlugins(BuildInfoPlugin).
settings(
buildInfoKeys := Seq[BuildInfoKey](name, version, scalaVersion, sbtVersion),
buildInfoOptions += BuildInfoOption.BuildTime,
buildInfoPackage := "com.planalytics",
buildInfoObject := "BuildInfo"
)
lazy val ingestion = Project(
id = "Ingestion-Engine",
base = file("ingestion"),
settings = Project.defaultSettings ++ sharedSettings ++ sparkDeps ++ akkaDeps ++ sharedDeps).
settings(
name := "Ingestion-Engine",
libraryDependencies ++= Seq(
"com.github.seratch" %% "awscala" % "0.5.+",
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % "test"
)
).dependsOn(util)
lazy val services = Project(
id = "Ingestion-Services",
base = file("services"),
settings = Project.defaultSettings ++ sharedSettings ++ sprayDeps ++ sharedDeps).
settings(
name := "Ingestion-Services",
libraryDependencies ++= Seq(
"io.spray" %% "spray-testkit" % sprayVersion % "test"
)
).dependsOn(ingestion)
lazy val util = Project(
id = "Ingestion-Util",
base = file("util"),
settings = Project.defaultSettings ++ sharedSettings ++ sparkDeps ++ sharedDeps).
settings(
name := "Ingestion-Utils",
libraryDependencies ++= Seq(
"net.ceedubs" %% "ficus" % "1.1.2"
),
dependencyOverrides ++= Set(
"com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)
)
I'm also open to any thoughts on my build in general, I feel like I'm doing some things wrong or repetitively, so any suggestions there is appreciated as well. Thanks!

Spray microservice assembly deduplicate

I'm using this template to develop a microservice:
http://www.typesafe.com/activator/template/activator-service-container-tutorial
My sbt file is like this:
import sbt._
import Keys._
name := "activator-service-container-tutorial"
version := "1.0.1"
scalaVersion := "2.11.6"
crossScalaVersions := Seq("2.10.5", "2.11.6")
resolvers += "Scalaz Bintray Repo" at "https://dl.bintray.com/scalaz/releases"
libraryDependencies ++= {
val containerVersion = "1.0.1"
val configVersion = "1.2.1"
val akkaVersion = "2.3.9"
val liftVersion = "2.6.2"
val sprayVersion = "1.3.3"
Seq(
"com.github.vonnagy" %% "service-container" % containerVersion,
"com.github.vonnagy" %% "service-container-metrics-reporting" % containerVersion,
"com.typesafe" % "config" % configVersion,
"com.typesafe.akka" %% "akka-actor" % akkaVersion exclude ("org.scala-lang" , "scala-library"),
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion exclude ("org.slf4j", "slf4j-api") exclude ("org.scala-lang" , "scala-library"),
"ch.qos.logback" % "logback-classic" % "1.1.3",
"io.spray" %% "spray-can" % sprayVersion,
"io.spray" %% "spray-routing" % sprayVersion,
"net.liftweb" %% "lift-json" % liftVersion,
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % "test",
"io.spray" %% "spray-testkit" % sprayVersion % "test",
"junit" % "junit" % "4.12" % "test",
"org.scalaz.stream" %% "scalaz-stream" % "0.7a" % "test",
"org.specs2" %% "specs2-core" % "3.5" % "test",
"org.specs2" %% "specs2-mock" % "3.5" % "test",
"com.twitter" %% "finagle-http" % "6.25.0",
"com.twitter" %% "bijection-util" % "0.7.2"
)
}
scalacOptions ++= Seq(
"-unchecked",
"-deprecation",
"-Xlint",
"-Ywarn-dead-code",
"-language:_",
"-target:jvm-1.7",
"-encoding", "UTF-8"
)
crossPaths := false
parallelExecution in Test := false
assemblyJarName in assembly := "santo.jar"
mainClass in assembly := Some("Service")
The project compiles fine!
But when I run assembly, the terminal show me this:
[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /path/.ivy2/cache/io.dropwizard.metrics/metrics-core/bundles/metrics-core-3.1.1.jar:com/codahale/metrics/ConsoleReporter$1.class
[error] /path/.ivy2/cache/com.codahale.metrics/metrics-core/bundles/metrics-core-3.0.1.jar:com/codahale/metrics/ConsoleReporter$1.class
What options do I have to fix it?
Thanks
The issue as it seems transitive dependency of the dependency is resulting with two different versions of metrics-core. The best thing to do would be to used the right library dependency so that you end up with a single version of this library. Please use https://github.com/jrudolph/sbt-dependency-graph , if it is difficult to figure out dependencies.
If it is not possible to get to a single version then you would most likely to go down exclude route . I assume, this only work, if there is compatibility between the all required versions.

sbt subproject cannot find it's dependencies

I have a project tree consisting of three projects A, B and C
B depends on A, and C depends on both A and B.
A and B are checked out in C's lib/ and both build fine using sbt compile
However, when I compile C, the build of B fails, complaining that it cannot find certain types/packages:
import org.scalatra.sbt._
import sbt.Keys._
import sbt._
object NwbApiBuild extends Build {
val Organization = "org.nwb"
val Name = "NWB API"
val Version = "0.1.0-SNAPSHOT"
val ScalaVersion = "2.10.3"
val ScalatraVersion = "2.3.0"
lazy val active_slick= Project (
"active-slick",
base = file("lib/active-slick")
)
lazy val slick_auth= Project (
"slick-auth",
base = file("lib/slick-auth")
)
lazy val project = Project (
"root",
file("."),
settings = Defaults.defaultSettings ++ ScalatraPlugin.scalatraWithJRebel ++ Seq(
organization := Organization,
name := Name,
version := Version,
scalaVersion := ScalaVersion,
resolvers += Classpaths.typesafeReleases,
libraryDependencies ++= Seq(
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"ch.qos.logback" % "logback-classic" % "1.0.6" % "runtime",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.8.v20121106" % "container",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" % "container;provided;test" artifacts (Artifact("javax.servlet", "jar", "jar")),
"com.typesafe.slick" %% "slick" % "2.0.2",
"mysql" % "mysql-connector-java" % "5.1.31",
"joda-time" % "joda-time" % "2.3",
"org.joda" % "joda-convert" % "1.5",
"com.github.tototoshi" %% "slick-joda-mapper" % "1.1.0",
"org.json4s" %% "json4s-native" % "3.2.10",
"org.json4s" %% "json4s-jackson" % "3.2.7",
"c3p0" % "c3p0" % "0.9.1.2"
)
)
) aggregate(active_slick, slick_auth) dependsOn(active_slick, slick_auth)
}
where slick auth has build file
import org.scalatra.sbt._
name := "slick-auth"
version := "0.0.1-SNAPSHOT"
scalaVersion := "2.10.3"
val ScalatraVersion = "2.3.0"
lazy val active_slick = Project(
"active-slick",
base = file("lib/active-slick")
)
lazy val root = Project(
"root",
file("."),
settings = Defaults.defaultSettings ++ ScalatraPlugin.scalatraSettings ++ Seq(
libraryDependencies ++= Seq(
"com.typesafe.slick" %% "slick" % "2.0.2",
"org.slf4j" % "slf4j-nop" % "1.6.4",
"org.scalatest" %% "scalatest" % "2.2.0" % "test",
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"ch.qos.logback" % "logback-classic" % "1.0.6" % "runtime",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.8.v20121106" % "container",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" % "container;provided;test" artifacts (Artifact("javax.servlet", "jar", "jar")),
"com.typesafe.slick" %% "slick" % "2.0.2",
"joda-time" % "joda-time" % "2.3",
"org.joda" % "joda-convert" % "1.5",
"com.github.tototoshi" %% "slick-joda-mapper" % "1.1.0",
"org.json4s" %% "json4s-native" % "3.2.10",
"org.json4s" %% "json4s-jackson" % "3.2.7",
"c3p0" % "c3p0" % "0.9.1.2"
)
)
).aggregate(active_slick).dependsOn(active_slick)
and active_slick:
name := "active-slick"
version := "0.0.1-SNAPSHOT"
scalaVersion := "2.10.3"
libraryDependencies ++= Seq(
"com.typesafe.slick" %% "slick" % "2.0.2",
"org.slf4j" % "slf4j-nop" % "1.6.4",
"org.scalatest" %% "scalatest" % "2.2.0" % "test",
"com.h2database" % "h2" % "1.3.166" % "test"
)
If you want to use another project as a dependency (rather than its binary version) you can use project references. There are two types of references, ProjectRef or a simpler version of the ProjectRef, which is RootProject.
You should change your build definition to reference slick_auth as
lazy val slick_auth = RootProject(file("lib/slick-auth"))
and active_slick as
lazy val active_slick = RootProject(file("lib/active-slick"))