sbt using different Scala versions in a multi project build - scala

I have a multi project which uses two versions of Scala. The Build.scala looks like this:
import sbt._
import Keys._
object Build extends sbt.Build {
lazy val root: Project = Project(
id = "scalacolliderugens",
base = file("."),
aggregate = Seq(gen, core)
)
lazy val gen = Project(...)
lazy val core = Project(...)
val ugenGenerator = TaskKey[Seq[File]]("ugen-generate", "Generate UGen class files")
def runUGenGenerator(source: File, cp: Seq[File]): Seq[File] = Nil // TODO
}
Where I have a pure source generating project using Scala 2.9.2:
lazy val gen = Project(
id = "scalacolliderugens-gen",
base = file("gen"),
settings = Project.defaultSettings ++ Seq(
scalaVersion := "2.9.2", // !!!
libraryDependencies ++= Seq(
"org.scala-refactoring" % "org.scala-refactoring_2.9.1" % "0.4.1"
),
libraryDependencies <+= scalaVersion { sv =>
"org.scala-lang" % "scala-compiler" % sv
}
)
)
And the actual project which incorporates the generated sources, compiling against Scala 2.10:
lazy val core = Project(
id = "scalacolliderugens-core",
base = file("core"),
settings = Project.defaultSettings ++ Seq(
scalaVersion := "2.10.0", // !!!
sourceGenerators in Compile <+= (ugenGenerator in Compile),
ugenGenerator in Compile <<=
(scalaSource in Compile, dependencyClasspath in Runtime in gen) map {
(src, cp) => runUGenGenerator(src, cp.files)
}
)
).dependsOn(gen)
When I compile this, I get sbt warnings:
[warn] Binary version (2.9.2) for dependency org.scala-lang#scala-library;2.9.2
[warn] in de.sciss#scalacolliderugens_2.10;1.0.1 differs from Scala binary \
version in project (2.10).
[info] Resolving org.scala-lang#scala-library;2.9.2 ...
[info] Done updating.
[warn] Binary version (2.9.2) for dependency org.scala-lang#scala-library;2.9.2
[warn] in de.sciss#scalacolliderugens-gen_2.10;1.0.1 differs from Scala binary \
version in project (2.10).
[warn] Binary version (2.9.2) for dependency org.scala-lang#scala-compiler;2.9.2
[warn] in de.sciss#scalacolliderugens-gen_2.10;1.0.1 differs from Scala binary \
version in project (2.10).
Should I be worried? Am I doing something bad here?

Add the following setting to the gen project.
scalaBinaryVersion:= CrossVersion.binaryScalaVersion("2.9.2")

OK...
I have multi-project SBT projects and they generally have SBT settings that are partly shared by all and partly specific to each sub-project. In my case the common settings include the Scala version to use.
You could structure you build specification the same way and put the Scala version setting in the sub-project-specific settings. If necessary, you could stratify the settings further if there are some settings common to all and some common to subsets of the sub-projects and other specific to each individual sub-project.

You cannot mix bytecode from different minor versions of Scala. I.e., code compiled by any 2.9.x series compiler is not compatible with that emitted by a 2.10.x (or 2.8.x) compiler.

Related

Pre-test SBT task: Unable to instantiate JDBC driver

I'm having trouble getting an SBT task to run migrations with Flyway; I get an exception when I run the task. Any ideas how I could fix it?
org.flywaydb.core.api.FlywayException: Unable to instantiate JDBC driver: org.postgresql.Driver => Check whether the jar file is present
The following code works, when I run it in BeforeAll, in my tests (ScalaTest), but does not work when I move it into an SBT task.
val flyway = Flyway
.configure()
.locations("filesystem:./**/resources/db/migrations/")
.dataSource("jdbc:postgresql://localhost:5432/my_database", "my_user", "secret")
.load()
flyway.clean()
flyway.migrate()
My /build.sbt file looks like this:
import org.flywaydb.core.Flyway
lazy val migrate = taskKey[Unit]("Migrate database")
lazy val migrateTask = Def.task {
println("Migrate")
val flyway = Flyway
.configure()
.locations("filesystem:./**/resources/db/migrations/")
.dataSource("jdbc:postgresql://localhost:5432/my_database", "my_user", "secret")
.load()
flyway.clean()
flyway.migrate()
}
val IntegrationTest = config("integration") extend Test
lazy val integrationTestSettings = inConfig(IntegrationTest)(Defaults.testSettings) ++ List(
IntegrationTest / fork := false,
IntegrationTest / parallelExecution := false,
IntegrationTest / sourceDirectory := baseDirectory.value / "src/test/integration",
IntegrationTest / test := {
(IntegrationTest / test) dependsOn migrateTask
}.value
)
lazy val root = Project(id = "hello", base = file("."))
.configs(Configs.all: _*)
.settings(
integrationTestSettings,
libraryDependencies += "org.scalatest" %% "scalatest" % "3.1.4",
)
And my /project/build.sbt looks like this:
libraryDependencies ++= List(
"org.flywaydb" % "flyway-core" % "7.6.0",
"org.postgresql" % "postgresql" % "42.2.19",
)
The versions I'm using are:
SBT: 1.4.5
Scala: 2.13.4
Flyway: 7.6.0
Does anyone have any ideas why I'm getting that error, and how I can fix it?
Any help would be greatly appreciated. Thanks :)
Searching on the Flyway repo, the error message is coming from here - https://github.com/flyway/flyway/blob/9033185ab8bfa56b0dae9136c04763cdccc50081/flyway-core/src/main/java/org/flywaydb/core/internal/jdbc/DriverDataSource.java#L165-L182 where it's trying load the database driver from the classloader. These ClassLoader techniques sometimes clash with the sbt set up layered ClassLoader to run sbt itself. That's my speculation on what's happening.
How do we work around this?
You said that running it as part the test worked, so maybe you could create a subproject for this purpose?
ThisBuild / scalaVersion := "2.13.4"
lazy val migrate = taskKey[Unit]("Migrate database")
lazy val root = (project in file("."))
.settings(
name := "hello",
migrate := (migrateProj / run).toTask("").value
)
// utility project to run database migration
lazy val migrateProj = (project in file("migrate"))
.settings(
libraryDependencies ++= List(
"org.flywaydb" % "flyway-core" % "7.6.0",
"org.postgresql" % "postgresql" % "42.2.19",
),
Compile / run / fork := true,
publish / skip := true,
)
migrate/Migrate.scala
object Migrate extends App {
println("migrate")
// rest of the code here...
}
Now you can run
sbt:flyway> migrate
[info] running (fork) Migrate
[info] migrate
[success] Total time: 4 s, completed Mar 6, 2021 9:03:07 PM
Details about layered ClassLoader
ClassLoader techniques sometimes clash with the sbt set up layered ClassLoader to run sbt itself. sbt-the-Bash-script allows users to choose the sbt version using project/build.properties, and Scala version using build.sbt. Both of these make sbt build declarative and repeatable, and generally a good thing. But how can sbt launcher written using Scala 2.10 launch sbt 1.4.x written using Scala 2.12, which then launch your Scala 2.13 application? Each of these boundary cross is done by creating a layered ClassLoader, like the movie Inception.

Error while using aspectj with Scala

I am having an application in scala. I need to use AOP for one of the functionality. I used the plugin sbt-aspectj . Everything is working fine when I run using the sbt console. However, I am not able to make it work when using the executable jar. I tried the the sample code provided in the sbt-aspect git page. But, I am getting the errors as
[warn] warning incorrect classpath: D:\source\jvm\modules\scala\frameworks\aspectjTracer\target\scala-2.11\classes
[warn] Missing message: configure.invalidClasspathSection in: org.aspectj.ajdt.ajc.messages
[error] error no sources specified
.
[trace] Stack trace suppressed: run 'last aspectjTracer/aspectj:ajc' for the full output.
[error] (aspectjTracer/aspectj:ajc) org.aspectj.bridge.AbortException: ABORT
[error] Expected project ID
[error] Expected configuration
[error] Expected ':' (if selecting a configuration)
[error] Expected key
[error] Not a valid key: aspectjTracker (similar: aspectjSource, aspectj-source, aspectjDirectory)
[error] last aspectjTracker/aspectj:ajc
[error]
My Build.scala is given below :
object frameworkBuild extends Build {
import Dependencies._
val akkaV = "2.3.6"
val sprayV = "1.3.1"
val musterV = "0.3.0"
val common_settings = Defaults.defaultSettings ++
Seq(version := "1.3-SNAPSHOT",
organization := "com.reactore",
scalaVersion in ThisBuild := "2.11.2",
scalacOptions ++= Seq("-unchecked", "-feature", "-deprecation"),
libraryDependencies := frameworkDependencies ++ testLibraryDependencies,
publishMavenStyle := true,
)
connectInput in run := true
lazy val aspectJTracer = Project(
"aspectjTracer",
file("aspectjTracer"),
settings = common_settings ++ aspectjSettings ++ Seq(
// input compiled scala classes
inputs in Aspectj <+= compiledClasses,
// ignore warnings
lintProperties in Aspectj += "invalidAbsoluteTypeName = ignore",
lintProperties in Aspectj += "adviceDidNotMatch = ignore",
// replace regular products with compiled aspects
products in Compile <<= products in Aspectj
)
)
// test that the instrumentation works
lazy val instrumented = Project(
"instrumented",
file("instrumented"),
dependencies = Seq(aspectJTracer),
settings = common_settings ++ aspectjSettings ++ Seq(
// add the compiled aspects from tracer
binaries in Aspectj <++= products in Compile in aspectJTracer,
// weave this project's classes
inputs in Aspectj <+= compiledClasses,
products in Compile <<= products in Aspectj,
products in Runtime <<= products in Compile
)
)
lazy val frameworks = Project(id = "frameworks", base = file("."), settings = common_settings).aggregate( core, baseDomain,aspectJTracer,instrumented)
lazy val core = Project(id = "framework-core", base = file("framework-core"), settings = common_settings)
lazy val baseDomain = Project(id = "framework-base-domain", base = file("framework-base-domain"), settings = common_settings).dependsOn(core,aspectJTracer,instrumented)
}
Does anyone know how to fix this? I posted this in the sbt-aspectj github page and waiting for a response there as well. But I am in a little hurry to fix this. Your help will be really appreciated.
Finally the problem is resolved. I had added javaagent in the build.scala. But, while running with the sbt-one-jar, it was not taking that jar. So I have manually provided the javaagent as the aspectweaver jar file and it worked. However, it is almost taking 3-4 minutes to start the jar file with the aspect.
Sometime it is even taking 15 min to start the jar file due to the aspectjwaver. I am not sure if this is problem with aspectj or sbt-one-jar, I guess its with the one-jar.
has anyone else faced the same issue ? I don't see any activity in sbt-one-jar, so asking it here.

Why does sbt report "not found: value PlayScala" with Build.scala while build.sbt works?

I am creating a multi-module sbt project, with following structure:
<root>
----build.sbt
----project
----Build.scala
----plugins.sbt
----common
----LoggingModule
LoggingModule is a Play Framework project, while common is a simple Scala project.
In plugins.sbt:
resolvers += "Typesafe repo" at "http://repo.typesafe.com/typesafe/releases/"
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.3.3")
While I have this in build.sbt, all works fine and it recognises PlayScala:
name := "Multi-Build"
lazy val root = project.in(file(".")).aggregate(common, LoggingModule).dependsOn(common, LoggingModule)
lazy val common = project in file("common")
lazy val LoggingModule = (project in file("LoggingModule")).enablePlugins(PlayScala)
However as soon I put this in project/Build.scala instead of `build.sbt' as follows:
object RootBuild extends Build {
lazy val root = project.in(file("."))
.aggregate(common, LoggingModule)
.dependsOn(common, LoggingModule)
lazy val common = project in file("common")
lazy val LoggingModule = (project in file("LoggingModule")).enablePlugins(PlayScala)
...//other settings
}
it generates error as:
not found: value PlayScala
lazy val LoggingModule = (project in file("LoggingModule")).enablePlugins(PlayScala)
^
How to solve the issue?
It's just a missing import.
In .sbt files, some things are automatically imported by default: contents of objects extending Plugin, and (>= 0.13.5) autoImport fields in AutoPlugins. This is the case of PlayScala.
In a Build.scala file, normal Scala import rules apply. So you have to import things a bit more explicitly. In this case, you need to import play.PlayScala (or use .enabledPlugins(play.PlayScala) directly).

Why does sbt console not see packages from subproject in multi-module project?

This is my project/Build.scala:
package sutils
import sbt._
import Keys._
object SutilsBuild extends Build {
scalaVersion in ThisBuild := "2.10.0"
val scalazVersion = "7.0.6"
lazy val sutils = Project(
id = "sutils",
base = file(".")
).settings(
test := { },
publish := { }, // skip publishing for this root project.
publishLocal := { }
).aggregate(
core
)
lazy val core = Project(
id = "sutils-core",
base = file("sutils-core")
).settings(
libraryDependencies += "org.scalaz" % "scalaz-core_2.10" % scalazVersion
)
}
This seems to be compiling my project just fine, but when I go into the console, I can't import any of the code that just got compiled?!
$ sbt console
scala> import com.github.dcapwell.sutils.validate.Validation._
<console>:7: error: object github is not a member of package com
import com.github.dcapwell.sutils.validate.Validation._
What am I doing wrong here? Trying to look at the usage, I don't see a way to say which subproject to load while in the console
$ sbt about
[info] Loading project definition from /src/sutils/project
[info] Set current project to sutils (in build file:/src/sutils/)
[info] This is sbt 0.13.1
[info] The current project is {file:/src/sutils/}sutils 0.1-SNAPSHOT
[info] The current project is built against Scala 2.10.3
[info] Available Plugins: org.sbtidea.SbtIdeaPlugin
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.3
There's the solution from #Alexey-Romanov to start the console task in the project the classes to import are in.
sbt sutils/console
There's however another solution that makes the root sutils project depend on the other core. Use the following snippet to set up the project - note dependsOn core that will bring the classes from the core project to sutils's namespace.
lazy val sutils = Project(
id = "sutils",
base = file(".")
).settings(
test := { },
publish := { }, // skip publishing for this root project.
publishLocal := { }
).aggregate(
core
).dependsOn core
BTW, you should really use a simpler build.sbt for your use case as follows:
scalaVersion in ThisBuild := "2.10.0"
val scalazVersion = "7.0.6"
lazy val sutils = project.in(file(".")).settings(
test := {},
publish := {}, // skip publishing for this root project.
publishLocal := {}
).aggregate(core).dependsOn(core)
lazy val core = Project(
id = "sutils-core",
base = file("sutils-core")
).settings(
libraryDependencies += "org.scalaz" %% "scalaz-core" % scalazVersion
)
You could make it even easier when you'd split the build to two build.sbts, each for the projects.

How to get list of dependency jars from an sbt 0.10.0 project

I have a sbt 0.10.0 project that declares a few dependencies somewhat like:
object MyBuild extends Build {
val commonDeps = Seq("commons-httpclient" % "commons-httpclient" % "3.1",
"commons-lang" % "commons-lang" % "2.6")
val buildSettings = Defaults.defaultSettings ++ Seq ( organization := "org" )
lazy val proj = Project("proj", file("src"),
settings = buildSettings ++ Seq(
name := "projname",
libraryDependencies := commonDeps, ...)
...
}
I wish to creat a build rule to gather all the jar dependencies of "proj", so that I can symlink them to a single directory.
Thanks.
Example SBT task to print full runtime classpath
Below is roughly what I'm using. The "get-jars" task is executable from the SBT prompt.
import sbt._
import Keys._
object MyBuild extends Build {
// ...
val getJars = TaskKey[Unit]("get-jars")
val getJarsTask = getJars <<= (target, fullClasspath in Runtime) map { (target, cp) =>
println("Target path is: "+target)
println("Full classpath is: "+cp.map(_.data).mkString(":"))
}
lazy val project = Project (
"project",
file ("."),
settings = Defaults.defaultSettings ++ Seq(getJarsTask)
)
}
Other resources
Unofficial guide to sbt 0.10.
Keys.scala defines predefined keys. For example, you might want to replace fullClasspath with managedClasspath.
This plugin defines a simple command to generate an .ensime file, and may be a useful reference.