Cannot import classes from one module to other one - Scala - scala

I have created a project with 3 different modules. First one is called http and second algebra. I have connected them into one in sbt file, but when I want to use classes from algebra in http then I cannot import them because they do not see each other.
This is my sbt file:
lazy val commonSettings = Seq(
libraryDependencies ++= Seq(
"org.typelevel" %% "cats-core" % CatsVersion,
"org.typelevel" %% "cats-effect" % "1.2.0",
"org.typelevel" %% "cats-tagless-macros" % "0.2.0",
"org.typelevel" %% "cats-mtl-core" % "0.5.0",
)
)
lazy val root = project.in(file(".")).aggregate(http, domain, algebra)
.settings(commonSettings)
.settings(libraryDependencies ++= Seq(
"org.tpolecat" %% "doobie-core" % DoobieVersion,
"org.tpolecat" %% "doobie-h2" % DoobieVersion,
"org.tpolecat" %% "doobie-scalatest" % DoobieVersion,
"org.tpolecat" %% "doobie-hikari" % DoobieVersion,
))
lazy val http = (project in file("http"))
.dependsOn(algebra)
.settings(commonSettings)
.settings(
name := "my-http",
libraryDependencies ++= Seq(
"io.circe" %% "circe-generic" % CirceVersion,
"io.circe" %% "circe-literal" % CirceVersion,
"io.circe" %% "circe-generic-extras" % CirceVersion,
"io.circe" %% "circe-parser" % CirceVersion,
"io.circe" %% "circe-java8" % CirceVersion,
"io.circe" %% "circe-config" % CirceConfigVersion,
"org.http4s" %% "http4s-blaze-server" % Http4sVersion,
"org.http4s" %% "http4s-circe" % Http4sVersion,
"org.http4s" %% "http4s-dsl" % Http4sVersion,
))
lazy val domain = project.in(file("domain"))
lazy val algebra = (project in file("algebra"))
.settings(commonSettings)
.settings(
name := "my-algebra",
)
I tried to refresh all projects but it did not work.
class MyRoutes[F[_]: Effect](services: MyService[F]) extends Http4sDsl[F]{...}
Class MyRoutes is in http module and MyService in algebra module. The error is Cannot find declaration to go to on MyService.
How can I fix it?

Ok, I solved this problem. It was my, stupid mistake. I have not marked directory as source root where is MyService. Because of this, in http module I could not see this class.

Related

New sbt application using the akka-http template, how to determine resolvers and add maven central?

I have a new sbt application that I built using the akka http g8 template.
I am trying to add reactivemongo 1.0 to my build and I am getting this error:
not found: https://repo1.maven.org/maven2/org/reactivemongo/reactivemongo_2.13/1.0/reactivemongo_2.13-1.0.pom
The documentation says this library is in maven central.
How can I determine which resolver my project is using by default currently in sbt?
Is it possible that this library is not built for scala 2.13.3 or 2.13.1?
How can I debug this type of error.
Thanks!
build.sbt:
import Dependencies._
lazy val akkaHttpVersion = "10.2.1"
lazy val akkaVersion = "2.6.10"
lazy val root = (project in file("."))
.settings(
inThisBuild(
List(
organization := "com.example",
scalaVersion := "2.13.3"
)
),
name := "akka-http",
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-http" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-actor-typed" % akkaVersion,
"com.typesafe.akka" %% "akka-stream" % akkaVersion,
"ch.qos.logback" % "logback-classic" % "1.2.3",
"com.softwaremill.macwire" %% "macros" % "2.3.3" % "provided",
"com.softwaremill.macwire" %% "util" % "2.3.3" % "provided",
"com.github.blemale" %% "scaffeine" % "3.1.0" % "compile",
"org.typelevel" %% "cats-core" % "2.1.1",
"com.lihaoyi" %% "scalatags" % "0.8.2",
"com.github.pureconfig" %% "pureconfig" % "0.13.0",
"org.reactivemongo" %% "reactivemongo" % "1.0",
"com.typesafe.akka" %% "akka-http-testkit" % akkaHttpVersion % Test,
"com.typesafe.akka" %% "akka-actor-testkit-typed" % akkaVersion % Test,
"org.scalatest" %% "scalatest" % "3.0.8" % Test
)
)
.enablePlugins(JavaAppPackaging)
Can you try replacing "org.reactivemongo" %% "reactivemongo" % "1.0" with "org.reactivemongo" %% "reactivemongo" % "1.0.0" % "provided"?
I copy it from Maven Repository https://mvnrepository.com/artifact/org.reactivemongo/reactivemongo_2.13/1.0.0

`java.lang.NoSuchMethodError: cats.FlatMap.map2` in runtime when using `.sequence`

I'm getting the following runtime error after migrating from cats v1.1.0 to v1.4.0 (An error arises from places where .sequence (cats.Traverse) is used).
The code looks like:
import cats.implicits._
import cats.effect.IO
List(1, 2, 3).map(x => IO(...)).sequence
java.lang.NoSuchMethodError: cats.FlatMap.map2$(Lcats/FlatMap;Ljava/lang/Object;Ljava/lang/Object;Lscala/Function2;)Ljava/lang/Object;
at cats.effect.IOLowPriorityInstances$IOEffect.map2(IO.scala:765)
...
Here is my build.sbt:
organization := "org.xxx"
name := "yyy"
version := "0.0.1"
scalaVersion := "2.12.9"
resolvers += Resolver.bintrayRepo("hseeberger", "maven")
resolvers ++= Seq("Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/")
lazy val doobieVersion = "0.5.3"
lazy val akkaHttpVersion = "10.1.1"
lazy val akkaVersion = "2.5.12"
lazy val catsVersion = "1.4.0"
lazy val circeVersion = "0.9.3"
lazy val doobieDeps = Seq(
"org.tpolecat" %% "doobie-core" % doobieVersion,
"org.tpolecat" %% "doobie-postgres" % doobieVersion,
"org.tpolecat" %% "doobie-scalatest" % doobieVersion,
"org.tpolecat" %% "doobie-hikari" % doobieVersion
)
lazy val catsDeps = Seq(
"org.typelevel" %% "cats-effect" % catsVersion,
"org.typelevel" %% "cats-core" % catsVersion
)
lazy val otherDeps = Seq(
"com.github.pureconfig" %% "pureconfig" % "0.9.1",
"org.scorexfoundation" %% "scrypto" % "2.1.1",
"de.heikoseeberger" %% "akka-http-circe" % "1.20.1",
"org.scalaj" %% "scalaj-http" % "2.4.0",
"org.flywaydb" % "flyway-core" % "5.1.1",
"com.github.blemale" %% "scaffeine" % "2.5.0",
("org.scorexfoundation" %% "sigma-state" % "master-2b4b07a1-SNAPSHOT")
.exclude("ch.qos.logback", "logback-classic")
.exclude("org.scorexfoundation", "scrypto"),
)
lazy val circeDeps = Seq(
"io.circe" %% "circe-core" % circeVersion,
"io.circe" %% "circe-parser" % circeVersion,
"io.circe" %% "circe-generic" % circeVersion
)
libraryDependencies ++= (otherDeps ++ doobieDeps ++ catsDeps ++ loggingDeps ++ akkaDeps ++ circeDeps ++ testDeps)
I've tried to run it different ways (idea, set run) and on different platforms - the result is always the same.
What could it be caused by?
Try to change versions to
"org.typelevel" %% "cats-effect" % "1.4.0",
"org.typelevel" %% "cats-core" % "1.6.1"
Your project seems to work with them.

Unable to create spark-warehouse directory using spark-2.3.0

I want to create a project with akka and spark. I added dependencies and some other dependencies too. Is these dependencies will cause any effect on using spark.
I have below sbt file
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-core" % "2.8.7"
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.8.7"
dependencyOverrides += "com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.8.7"
lazy val commonSettings = Seq(
organization := "com.bitool.analytics",
scalaVersion := "2.11.12",
libraryDependencies ++= Seq(
"org.scala-lang.modules" %% "scala-async" % "0.9.6",
"com.softwaremill.macwire" %% "macros" % "2.3.0",
"com.softwaremill.macwire" %% "macrosakka" % "2.3.0",
"com.typesafe.akka" %% "akka-http" % "10.0.6",
"io.swagger" % "swagger-jaxrs" % "1.5.19",
"com.github.swagger-akka-http" %% "swagger-akka-http" % "0.9.1",
"io.circe" %% "circe-generic" % "0.8.0",
"io.circe" %% "circe-literal" % "0.8.0",
"io.circe" %% "circe-parser" % "0.8.0",
"io.circe" %% "circe-optics" % "0.8.0",
"org.scalafx" %% "scalafx" % "8.0.144-R12",
"org.scalafx" %% "scalafxml-core-sfx8" % "0.4",
"org.apache.spark" %% "spark-core" % "2.3.0",
"org.apache.spark" %% "spark-sql" % "2.3.0",
"org.apache.spark" %% "spark-hive" % "2.3.0",
"org.scala-lang" % "scala-xml" % "2.11.0-M4",
"mysql" % "mysql-connector-java" % "6.0.5"
)
)
lazy val root = (project in file(".")).
settings(commonSettings: _*).
settings(
name := "BITOOL-1.0"
)
ivyScala := ivyScala.value map {
_.copy(overrideScalaVersion = true)
}
fork in run := true
and below is my spark code
private val warehouseLocation = new File("spark-warehouse").getAbsolutePath
val conf = new SparkConf()
conf.setMaster("local[4]")
conf.setAppName("Bitool")
conf.set("spark.sql.warehouse.dir", warehouseLocation)
val SPARK = SparkSession
.builder().config(conf).enableHiveSupport()
.getOrCreate()
val SPARK_CONTEXT = SPARK.sparkContext
When I trying to execute this, It is creating metastore_db folder but spark-warehouse folder is not creating.
This directory is not created by getOrCreate. You can check it in the Spark source code: getOrCreate delegates its actions to SparkSession.getOrCreate, which is just a setter. All the internal tests and CliSuite use a snippet like this to prematurely initialize the dir: val warehousePath = Utils.createTempDir()
Instead, in the actual user code, you have to perform at least one data modification operation to materialize your warehouse directory. Try running something like that just after your code and check warehouse directory on the hard drive again:
import SPARK.implicits._
import SPARK.sql
sql("DROP TABLE IF EXISTS test")
sql("CREATE TABLE IF NOT EXISTS test (key INT, value STRING) USING hive")

Spray microservice assembly deduplicate

I'm using this template to develop a microservice:
http://www.typesafe.com/activator/template/activator-service-container-tutorial
My sbt file is like this:
import sbt._
import Keys._
name := "activator-service-container-tutorial"
version := "1.0.1"
scalaVersion := "2.11.6"
crossScalaVersions := Seq("2.10.5", "2.11.6")
resolvers += "Scalaz Bintray Repo" at "https://dl.bintray.com/scalaz/releases"
libraryDependencies ++= {
val containerVersion = "1.0.1"
val configVersion = "1.2.1"
val akkaVersion = "2.3.9"
val liftVersion = "2.6.2"
val sprayVersion = "1.3.3"
Seq(
"com.github.vonnagy" %% "service-container" % containerVersion,
"com.github.vonnagy" %% "service-container-metrics-reporting" % containerVersion,
"com.typesafe" % "config" % configVersion,
"com.typesafe.akka" %% "akka-actor" % akkaVersion exclude ("org.scala-lang" , "scala-library"),
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion exclude ("org.slf4j", "slf4j-api") exclude ("org.scala-lang" , "scala-library"),
"ch.qos.logback" % "logback-classic" % "1.1.3",
"io.spray" %% "spray-can" % sprayVersion,
"io.spray" %% "spray-routing" % sprayVersion,
"net.liftweb" %% "lift-json" % liftVersion,
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % "test",
"io.spray" %% "spray-testkit" % sprayVersion % "test",
"junit" % "junit" % "4.12" % "test",
"org.scalaz.stream" %% "scalaz-stream" % "0.7a" % "test",
"org.specs2" %% "specs2-core" % "3.5" % "test",
"org.specs2" %% "specs2-mock" % "3.5" % "test",
"com.twitter" %% "finagle-http" % "6.25.0",
"com.twitter" %% "bijection-util" % "0.7.2"
)
}
scalacOptions ++= Seq(
"-unchecked",
"-deprecation",
"-Xlint",
"-Ywarn-dead-code",
"-language:_",
"-target:jvm-1.7",
"-encoding", "UTF-8"
)
crossPaths := false
parallelExecution in Test := false
assemblyJarName in assembly := "santo.jar"
mainClass in assembly := Some("Service")
The project compiles fine!
But when I run assembly, the terminal show me this:
[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /path/.ivy2/cache/io.dropwizard.metrics/metrics-core/bundles/metrics-core-3.1.1.jar:com/codahale/metrics/ConsoleReporter$1.class
[error] /path/.ivy2/cache/com.codahale.metrics/metrics-core/bundles/metrics-core-3.0.1.jar:com/codahale/metrics/ConsoleReporter$1.class
What options do I have to fix it?
Thanks
The issue as it seems transitive dependency of the dependency is resulting with two different versions of metrics-core. The best thing to do would be to used the right library dependency so that you end up with a single version of this library. Please use https://github.com/jrudolph/sbt-dependency-graph , if it is difficult to figure out dependencies.
If it is not possible to get to a single version then you would most likely to go down exclude route . I assume, this only work, if there is compatibility between the all required versions.

sbt subproject cannot find it's dependencies

I have a project tree consisting of three projects A, B and C
B depends on A, and C depends on both A and B.
A and B are checked out in C's lib/ and both build fine using sbt compile
However, when I compile C, the build of B fails, complaining that it cannot find certain types/packages:
import org.scalatra.sbt._
import sbt.Keys._
import sbt._
object NwbApiBuild extends Build {
val Organization = "org.nwb"
val Name = "NWB API"
val Version = "0.1.0-SNAPSHOT"
val ScalaVersion = "2.10.3"
val ScalatraVersion = "2.3.0"
lazy val active_slick= Project (
"active-slick",
base = file("lib/active-slick")
)
lazy val slick_auth= Project (
"slick-auth",
base = file("lib/slick-auth")
)
lazy val project = Project (
"root",
file("."),
settings = Defaults.defaultSettings ++ ScalatraPlugin.scalatraWithJRebel ++ Seq(
organization := Organization,
name := Name,
version := Version,
scalaVersion := ScalaVersion,
resolvers += Classpaths.typesafeReleases,
libraryDependencies ++= Seq(
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"ch.qos.logback" % "logback-classic" % "1.0.6" % "runtime",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.8.v20121106" % "container",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" % "container;provided;test" artifacts (Artifact("javax.servlet", "jar", "jar")),
"com.typesafe.slick" %% "slick" % "2.0.2",
"mysql" % "mysql-connector-java" % "5.1.31",
"joda-time" % "joda-time" % "2.3",
"org.joda" % "joda-convert" % "1.5",
"com.github.tototoshi" %% "slick-joda-mapper" % "1.1.0",
"org.json4s" %% "json4s-native" % "3.2.10",
"org.json4s" %% "json4s-jackson" % "3.2.7",
"c3p0" % "c3p0" % "0.9.1.2"
)
)
) aggregate(active_slick, slick_auth) dependsOn(active_slick, slick_auth)
}
where slick auth has build file
import org.scalatra.sbt._
name := "slick-auth"
version := "0.0.1-SNAPSHOT"
scalaVersion := "2.10.3"
val ScalatraVersion = "2.3.0"
lazy val active_slick = Project(
"active-slick",
base = file("lib/active-slick")
)
lazy val root = Project(
"root",
file("."),
settings = Defaults.defaultSettings ++ ScalatraPlugin.scalatraSettings ++ Seq(
libraryDependencies ++= Seq(
"com.typesafe.slick" %% "slick" % "2.0.2",
"org.slf4j" % "slf4j-nop" % "1.6.4",
"org.scalatest" %% "scalatest" % "2.2.0" % "test",
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"ch.qos.logback" % "logback-classic" % "1.0.6" % "runtime",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.8.v20121106" % "container",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" % "container;provided;test" artifacts (Artifact("javax.servlet", "jar", "jar")),
"com.typesafe.slick" %% "slick" % "2.0.2",
"joda-time" % "joda-time" % "2.3",
"org.joda" % "joda-convert" % "1.5",
"com.github.tototoshi" %% "slick-joda-mapper" % "1.1.0",
"org.json4s" %% "json4s-native" % "3.2.10",
"org.json4s" %% "json4s-jackson" % "3.2.7",
"c3p0" % "c3p0" % "0.9.1.2"
)
)
).aggregate(active_slick).dependsOn(active_slick)
and active_slick:
name := "active-slick"
version := "0.0.1-SNAPSHOT"
scalaVersion := "2.10.3"
libraryDependencies ++= Seq(
"com.typesafe.slick" %% "slick" % "2.0.2",
"org.slf4j" % "slf4j-nop" % "1.6.4",
"org.scalatest" %% "scalatest" % "2.2.0" % "test",
"com.h2database" % "h2" % "1.3.166" % "test"
)
If you want to use another project as a dependency (rather than its binary version) you can use project references. There are two types of references, ProjectRef or a simpler version of the ProjectRef, which is RootProject.
You should change your build definition to reference slick_auth as
lazy val slick_auth = RootProject(file("lib/slick-auth"))
and active_slick as
lazy val active_slick = RootProject(file("lib/active-slick"))