How to use SBT to run ScalaTest tests against a fat jar? - scala

I have a simple SBT project, consisting of some Scala code in src/main/scala and some test code in src/test/scala. I use the sbt-assembly plugin to create a fat jar for deployment onto remote systems. The fat jar includes all the dependencies of the Scala project, including the Scala runtime itself. This all works great.
Now I'm trying to figure out a way I can run the Scala tests against the fat jar. I tried the obvious thing, creating a new config extending the Test config and modifying the dependencyClasspath to be the fat JAR instead of the default value, however this fails because (I assume because) the Scala runtime is included in the fat jar and collides somehow with the already-loaded Scala runtime.
My solution right now works but it has serious drawbacks. I just use Fork.java to invoke Java on the org.scalatest.tools.Runner runner with a classpath set to include the test code and the fat jar and all of the test dependencies. The downside is that none of the SBT test richness works, there's no testQuick, there's not testOnly, and the test failure reporting is on stdout.
My question boils down to this: how does one use SBT's test commands to run tests when those tests are dependent not on their corresponding SBT compile output, but on a fat JAR file which itself includes all the Scala runtimes?

This is what I landed on (for specs2, but can be adapted). This is basically what you said was your Fork solution, but I figured I'd leave this here in case someone wanted to know what that might be. Unfortunately I don't think you can run this "officially" as a SBT test runner. I should also add that you still want Fork.java even though this is Scala, because Fork.scala depends on a runner class that I don't seem to have.
test.sbt (or build.sbt, if you want to put a bunch of stuff there - SBT reads all .sbt files in the root if you want to organize):
// Set up configuration for building a test assembly
Test / assembly / assemblyJarName := s"${name.value}-test-${version.value}.jar"
Test / assembly / assemblyMergeStrategy := (assembly / assemblyMergeStrategy).value
Test / assembly / assemblyOption := (assembly / assemblyOption).value
Test / assembly / assemblyShadeRules := (assembly / assemblyShadeRules).value
Test / assembly / mainClass := Some("org.specs2.runner.files")
Test / test := {
(Test / assembly).value
val assembledFile: String = (Test / assembly / assemblyOutputPath).value.getAbsolutePath
val minimalClasspath: Seq[String] = (Test / assembly / fullClasspath).value
.filter(_.metadata.get(moduleID.key).get.organization.matches("^(org\\.(scala-lang|slf4j)|log4j).*"))
.map(_.data.getAbsolutePath)
val runClass: String = (Test / assembly / mainClass).value.get
val classPath: Seq[String] = Seq(assembledFile) ++ minimalClasspath
val args: Seq[String] = Seq("-cp", classPath.mkString(":"), runClass)
val exitCode = Fork.java((Test / assembly / forkOptions).value, args)
if (exitCode != 0) {
throw new TestsFailedException()
}
}
Test / assembly / test := {}
Change in build.sbt:
lazy val root = (project in file("."))
.settings(/* your original settings are here */)
.settings(inConfig(Test)(baseAssemblySettings): _*) // enable assembling in test

Related

Set mainClass for sbt native packager Universal

I have a project that has the following build.sbt:
addCommandAlias("package", "dist")
lazy val actual = (project in file("."))
.enablePlugins(UniversalPlugin, JavaServerAppPackaging)
.settings(
name := "DeployerPod",
mainClass := Some("com.myself.executable.Runner"),
Compile / mainClass := Some("com.myself.executable.Runner"),
Compile / run / mainClass := Some("com.myself.utils.Pipeline"),
Universal / mainClass := Some("com.myself.executable.Runner"),
Universal / compile / mainClass := Some("com.myself.executable.Runner"),
)
We have a CICD which runs a Dockerfile.
There I have sbt run as one of the steps, which will execute com.myself.utils.Pipeline class to run a Scala class and do the pre requisites for the pipeline.
As one of the last sbt based steps, I'm also running sbt package, which eventually runs an sbt dist command. At this point, inside the extracted ZIP's bin folder, I see two BAT files corresponding to the two main classes. Unfortunately I only want the Runner class BAT instead of Pipeline BAT.
For this I tried running sbt package -main com.myself.executable.Runner but that failed saying Not a valid command: -
Is there a way I can specify the mainClass only for this Universal plugin somehow? Because the way I've tried in my build.sbt doesn't seem to work.

How do I make sbt include non-Java sources to published artifact?

How do I make sbt include non-Java sources to published artifact ?
I'm using Kotlin plugin and can't figure out how to force sbt to include .kt file into published source jar. It only includes .java files.
A lot of people online suggest adding following code to sbt script but it doesn't help
mappings in (Compile, packageSrc) ++= {
val base = (sourceManaged in Compile).value
val files = (managedSources in Compile).value
files.map { f => (f, f.relativeTo(base).get.getPath) }
},
I also tried
includeFilter in (Compile, packageSrc) := "*.scala" || "*.java" || "*.kt",
Here is output of some variables in sbt console
sbt:collections> show unmanagedSourceDirectories
[info] * /home/expert/work/sideprojects/unoexperto/extensions-collections/src/main/scala
[info] * /home/expert/work/sideprojects/unoexperto/extensions-collections/src/main/java
[info] * /home/expert/work/sideprojects/unoexperto/extensions-collections/src/main/kotlin
sbt:collections> show unmanagedSources
[info] * /home/expert/work/sideprojects/unoexperto/extensions-collections/src/main/java/com/walkmind/extensions/collections/TestSomething.java
which plugin you use for kotlin?
https://github.com/pfn/kotlin-plugin has the option kotlinSource to configure where the source directory is located.
sbt packageBin compiled kotlin files and include them to output jar.
build.sbt
// define kotlin source directory
kotlinSource in Compile := baseDirectory.value / "src/main/kotlin",
src/main/kotlin/org.test
package org.test
fun main(args: Array<String>) {
println("Hello World!")
}
console
sbt compile
sbt packageBin
target/scala-2.13
jar include MainKt.class
and folder org/test contains MainKt.class too.
would this solve your problem?
I found a workaround for this in my project https://github.com/makiftutuncu/e. I made following: https://github.com/makiftutuncu/e/blob/master/project/Settings.scala#L105
Basically, I added following setting in SBT to properly generate sources artifact:
// Include Kotlin files in sources
packageConfiguration in Compile := {
val old = (packageConfiguration in Compile in packageSrc).value
val newSources = (sourceDirectories in Compile).value.flatMap(_ ** "*.kt" get)
new Package.Configuration(
old.sources ++ newSources.map(f => f -> f.getName),
old.jar,
old.options
)
}
For the documentation artifact, I added Gradle build to my Kotlin module. I set it up as shown here https://github.com/makiftutuncu/e/blob/master/e-kotlin/build.gradle.kts. This way, I make Gradle build generate the Dokka documentation. And finally, added following setting in SBT to run Gradle while building docs:
// Delegate doc generation to Gradle and Dokka
doc in Compile := {
import sys.process._
Process(Seq("./gradlew", "dokkaJavadoc"), baseDirectory.value).!
target.value / "api"
}
I admit, this is a lot of work just to get 2 artifacts but it did the trick for me. 🤷🏻 Hope this helps.

SBT plugin how to make a source generator dependent on project's sources?

I'm trying to create a source generator in a SBT plugin that generate code based on the project's sources.
I tried something like this:
sourceGenerators in Compile += (sources in Compile) map { sources => doSomethingWithSources(sources) }
Unfortunately, SBT does not want to load this plugin due to the fact that there exists circular dependency.
Due to this fact I've created another task like this:
lazy val myTask = TaskKey[Unit]("myTask", "Do stuff")
This tasks actually depends on the sources value and generates the files.
Later I override the projectSettings value and add this:
myTask in Compile := {
val sourcesValue = (sources in Compile).value
doSomethingWithSources(sourcesValue)
},
sourcesGenerators in Compile += Def.task(Seq(new File("path/to/myGeneratedSource.scala"))).taskValue
I add this task as the dependency to the compile task in the build.sbt of the project that I want my plugin to do stuff like this:
compile in Compile <<= (compile in Compile) dependsOn (myTask in Compile)
While it works (the file is generated), when I launch the sbt command sbt run, it creates the file but does not compile it.
What is more, when I run just sbt compile run, it compiles only the project on the first (compile) task and generates my source and then on run part it compiles the generated source - so, in matter of speaking, it does somehow work, but it needs two compilations.
I'd like to ask if there is a simpler way to do this and, if not, how to make it work in only one compilation.

sbt-assembly include test classes

I follow sbt-assembly : including test classes from a config described in https://github.com/sbt/sbt-assembly that work ok doing assembly
When I load sbt I get
assembly.sbt:5: error: reference to jarName is ambiguous;
it is imported twice in the same scope by
import sbtassembly.AssemblyKeys._
and import _root_.sbtassembly.AssemblyPlugin.autoImport._
jarName in (Test, assembly) := s"${name.value}-test-${version.value}.jar"
^
So, I comment import line and run sbt:assembly but that begin the test but dont generate any -test-.jar.
Any one know how to generate the jar that include the test classes?
Thanks
I had to remove this line (I think it is now autoimported based on https://github.com/sbt/sbt-assembly/blob/546d200477b64e2602beeb65bfa04306122cd9f5/Migration.md)
import sbtassembly.AssemblyKeys._
And I added the rest (i.e. the two lines below) to build.sbt instead of assembly.sbt:
Project.inConfig(Test)(baseAssemblySettings)
jarName in (Test, assembly) := s"${name.value}-test-${version.value}.jar"
After taking those steps, test:assembly does produce a test jar for me however I expected the jar to only include test classes (similar to test:package), but it seems to include non-test classes as well. In other words, if I have src/main/scala/Foo.scala and src/test/scala/FooTest.scala then I thought that the jar produced by test:assembly would only include FooTest.class but it seems to also include Foo.class. Hopefully that's not an issue for you as I'm not yet sure how to workaround that.
EDIT: If you want the jar to only include classes from src/test (like I did), then you can add the following to your build.sbt to filter out everything else that may be on your classpath:
fullClasspath in (Test, assembly) := {
val cp = (fullClasspath in (Test, assembly)).value
cp.filter({x => x.data.getPath.contains("test-classes")})
}
This works for me:
lazy val root = project.settings(
assembly / fullClasspath := (assembly / fullClasspath).value ++ (Test / fullClasspath).value
)

How to configure sbt to load resources when running application?

My code (Java) reads an image from jar:
Main.class.getResourceAsStream("/res/logo.png")
Everything runs fine (if I start the app after packaging it into a jar). But when I run it using sbt's run task, it returns me null instead of needed stream.
Running this from sbt console also gives null:
getClass.getResourceAsStream("/res/logo.png")
Is there a way to tell sbt to put my resources on classpath?
EDIT:
I set the resources dir to be same as source dir:
build.sbt:
resourceDirectory <<= baseDirectory { _ / "src" }
When I loaded sbt's `console' and ran the following:
classOf[Main].getProtectionDomain().getCodeSource()
I got the location of my classes, but it does not contain neither res folder nor any of my resource files.
Seems that sbt copies resources only to the resulting jar, and does not copy them to classes dir. Should I modify compile task to move these resources files to classes dir?
EDIT2:
Yes, when I manually copy the resource file to classes dir, I can easily access it from console. So, how should I automate this process?
EDIT3:
It seems that sbt is just unable to see my resource folder - it does not add files to resulting jar file, actually!
Solution:
resourceDirectory in Compile <<= baseDirectory { _ / "src" }
I can't give you a full solution right now, but there is a setting called resourceDirectories to which you could add the res folder.
[EDIT]
For me it didn't work also if the resource was in the standard resource folder. Please try it that way:
Main.class.getClassLoader().getResourceAsStream("icon.png")
[EDIT2] This is the full build script (build.scala) which works if your resource is in src/main/java:
import sbt._
import Keys._
object TestBuild extends Build {
lazy val buildSettings = Seq(
organization := "com.test",
version := "1.0-SNAPSHOT",
scalaVersion := "2.9.1"
)
lazy val test = Project(
id = "test",
base = file("test"),
settings = Defaults.defaultSettings ++ Seq(resourceDirectory in Compile <<= javaSource in Compile)
)
}