I'm attempting to compile and run scala code that generates a file in SBT (in this case its a swagger file).
the following sbt executes fine. The jar is built and executed and the swagger.zip file the execution creates is in the target directory. I can't, however, seem to get the zip file to get published to my artifactory repo like my standard jar files would.
Any idea on what I'm missing?
publishArtifact in (Compile, packageBin) := false
publishArtifact in run := true
val myZipTask = taskKey[File]("swagger-zip")
myZipTask := {
file("swagger.zip")
}
addArtifact(Artifact("swagger", "zip", "zip"), myZipTask )
The run command doesn't trigger any publishing tasks, as that's not what it's for. I think you have tried to turn on publishing when run is called with "publishArtifact in run := true", but that code is not sufficient to achieve that; you'd need to attach many other Tasks to that command. That's not what you want anyway.
Your case fits entirely within the expected usage of the addArtifact helper (see the docs, and the code)
As you need to run code to generate artefacts, you should provide that code as a Task argument to the addArtifact helper, i.e.
val myZipTask = taskKey[File]("return the swagger-zip file")
val runZipCodeTask = taskKey[Unit]("run the swagger-zip code")
// See http://www.scala-sbt.org/0.13.2/docs/faq.html, “How can I create a custom run task, in addition to run?”
// and https://stackoverflow.com/questions/23409993/defining-sbt-task-that-invokes-method-from-project-code
fullRunTask(runZipCodeTask, Compile, "ZipGeneratorMainClass")
myZipTask := {
runZipCodeTask.value
file("target/swagger/swagger.zip")
}
addArtifact(Artifact("swagger", "zip", "zip"), myZipTask)
Then run "sbt publish"
Working demo
See https://gist.github.com/RichardBradley/5384a5e0da2427df237f42fe512b30b8 for a working demo.
Note about scopes - consider generating this file in a meta-project, not the main project
See https://stackoverflow.com/a/23416018/8261
Related
For a project based on Play with sbt, I'd like to have multiple flavors for test runs, using different configuration files. Motivation is being able to run tests either against a local or a remote database.
There's already a custom config file speicified for general test runs (in build.sbt):
javaOptions in Test += "-Dconfig.file=conf/application.test.conf"
Now I would like to have another command where the same tests run against some configuration file conf/application.test-ci.conf.
Approaches tried so far
Adding a command alias
addCommandAlias("test-ci", ";test -Dconfig.file=conf/application.test-ci.conf")
This fails with an error message of a missing semicolon (;), indicating that sbt interprets the resulting command line as multiple commands, but I don't understand why.
Extend Test
lazy val CITest = config("ci") extend Test
lazy val config = (project in file(".")).enablePlugins(PlayScala)
.configs(CITest)
.settings(inConfig(CITest)(Defaults.testTasks): _*)
.settings(
javaOptions in CITest += "-Dconfig.file=conf/application.test-ci.conf"
)
javaOptions in CITest -= "-Dconfig.file=conf/application.test.conf"
I'm don't fully understand what this is doing, but it always seems to pick up the other test configuration file.
How can I specify multiple test setups picking up different configuration files?
Try first applying the setting via set command and then follow up with test task like so
addCommandAlias(
"test-ci",
""";set Test/javaOptions ++= Seq("-Dconfig.file=conf/application.test.con"); test"""
)
Notice how ; separates the set from test.
Another approach is to modify the setting based on the environment. Usually there is some environment variable set on CI like CI or BUILD, so you can modify javaOptions conditionally (without any custom configurations):
Test/javaOptions ++= {
if (sys.env.get("CI").isEmpty) Seq.empty
else Seq("-Dconfig.file=conf/application.test-ci.conf")
}
Note: Test/javaOptions is the new syntax for javaOptions in Test (since sbt 1)
I am trying to define a separate package task without modifying the original task in compile configuration. This new task will package only a subset of classes conforming an API which we need to be able to share with other teams so they can write plugins for our application. So the end result will be two jars, one with the full application and a second one with a subset of the classes.
I approached this problem by creating a different configuration which I called pluginApi and would redefine the packageBin task within this new configuration so it does not change the original definition of packageBin. This idea was taken from here:
How to create custom "package" task to jar up only specific package in SBT?
In my build.stb I have:
lazy val PluginApi = config("pluginApi") extend(Compile) describedAs("Custom plugin api configuration")
lazy val root = project in file(".") overrideConfigs (PluginApi)
This effectively creates my new configuration and I can call
sbt pluginApi:packageBin
Which generates the complete jar in the same way as compile:packageBin would do. I then try to modify the mappings in the new packageBin task with:
mappings in (PluginApi, packageBin) ~= { (ms: Seq[(File, String)]) =>
ms filter { case (file, toPath) =>
toPath.startsWith("some/path/defining/api")
}
}
but this has no effect. I think the reason is because the call to pluginApi:packageBin is delegated to compile:packageBin rather than it being a cloned task.
I can redefine a new packageBin within the new scope like:
packageBin in PluginApi := {
}
However I would have to rewrite all packageBin functionality instead of reusing existing code. Also, in case that rewriting is unavoidable I am not sure how that implementation would be.
Could somebody provide an example about how to achieve this?
You could have it done as follows
lazy val PluginApi = config("pluginApi").extend(Compile)
inConfig(PluginApi)(Defaults.compileSettings) // you have to have standard
mappings in (PluginApi, packageBin) := {
val original = (mappings in (PluginApi, packageBin)).value
original.filter { case (file, toPath) => toPath.startsWith("some/path/defining/api") }
}
unmanagedSourceDirectories in PluginApi := (unmanagedSourceDirectories in Compile).value
Note that, if you keep your sources in src/main/scala you'll have to override unmanagedSourceDirectories in the newly created configuration.
Normally the unmanagedSourceDirectories contains the configuration name. E.g. src/pluginApi/scala or src/pluginApi/java.
I have had similar problems (with more than one jar per project). Our project uses ant - here you can do it, you just will repeat yourself a lot.
However, I have come to the conclusion that this scenario (2 JARs for one project) actually can be simplified by splitting the project - i.e. making 2 modules out of it.
This way, I don't have to "fight" tools which assume project==artifact (like sbt, maybe maven?, IDEA's default setting,...).
As a bonus point the compiler helps me to verify that my dependencies are correct, i.e. that I did not accidentally make my API package depend on the implementation package - when compiling everything together and only splitting classes apart in the JAR step, you do run the risk of getting an invalid dependency in your setup which you would only see when testing, because during compile time everything is compiled together.
I have an sbt project with a few subprojects, each of which publishes some artifacts and has a fairly extensive test suite.
When I run the build on my CI server, I want to publish the artifacts to a staging location and run the tests after the publishing task. Since others may want the artifacts, I'd like to tell sbt that I want it to build all the artifacts for all subprojects, then run all the tests, since by default it seems to run them interleaved in an unspecified order.
I have a ScopeFilter giving me access to all my subprojects, so I can make my ciBuild task depend on something like the following
(test in Test).all(subprojectScopeFilter).dependsOn(myArtifactsTask.all(subprojectScopeFilter))`
However, that doesn't seem to have any real effect on the order, and I definitely see some subprojects running tests before others have run their myArtifactsTask. I'm guessing that I don't fully understand how all works and it might be saying that each independent subproject's test task depends on that same subproject's myArtifactsTask? If that's the case, how can I specify what I want? Is it documented somewhere that I've missed? The manual describes the basics of all but not how it interacts with other constructs.
SBT will resolve automatically the order between task and projects and build them in that order.
What you could do is - let's assume you have three projects. Root and two sub-projects. I assume that the key myArtifactTask is defined in the root.
project/Build.scala
object MyBuild extends Build {
val myArtifactTask = TaskKey[Unit]("my-artifact-task", "My Artifact Task")
}
The myArtifactTask is implemented in both sub-projects.
subproject-a/build.sbt
myArtifactTask := {
println("myArtifactTask:project-a")
}
subproject-a/build.sbt
myArtifactTask := {
println("myArtifactTask:project-b")
}
What you want to do is to define your root's build.sbt in a way that it calls myArtifactTask in both projects. Then you could define new task testedArtifact which would depend on myArtifactTask.
build.sbt
lazy val testedArtifact = taskKey[Unit]("Runs myArtifactTask followed by tests")
lazy val inAnyProjectButRoot: ScopeFilter = ScopeFilter (
inAnyProject -- inProjects(ThisProject)
)
myArtifactTask := {
myArtifactTask.all(inAnyProjectButRoot).value
}
testedArtifact := {
(test in Test).all(anyProjectButRoot).value
}
testedArtifact <<= testedArtifact.dependsOn(myArtifactTask)
Now calling testedArtifactin the root project will first call all myArtifactTasks in sub-projects followed by tests.
I am trying to publish to a repository the zip file generated by the sbt-native-packager plugin through universal:packageBin task.
I configured my project like this:
publishTo := Some("Repo" at "http://repo")
publishMavenStyle := true
packagerSettings
packageArchetype.java_application
I am struggling trying to create a new sbt task (named publishZip) using publish task and packageBin task to publish the zip file. How can I achieve this ?
Add the following line to your sbt build (around packagerSettings should be fine)
deploymentSettings
Depending on what you want to do you may not need to define the publishZip task you could run
sbt universal:publish which should only publish the zip
redefine publish so it depends on universal:publish which would publish all the projects artifacts
publish <<= publish.dependsOn(publish in config("universal"))
Then run sbt publish.
For completeness sake deploymentSettings (and packagerSettings) come from com.typesafe.sbt.SbtNativePackager which is useful to know if you use a scala build :)
The deploymentSettings worked however I wanted to refine the settings. It seems there were several issues going on. I and finally came up with the following solution:
//--use sbt-native-packager default java application
packageArchetype.java_application
//--a dummy task to hold the result of the universal:packageBin to stop the circular dependency issue
val packageZip = taskKey[File]("package-zip")
//--hard coded result of "universal:packageBin"
packageZip := (baseDirectory in Compile).value / "target" / "universal" / (name.value + "-" + version.value + ".zip")
//--label the zip artifact as a zip instead of the default jar
artifact in (Universal, packageZip) ~= { (art:Artifact) => art.copy(`type` = "zip", extension = "zip") }
//--add the artifact so it is included in the publishing tasks
addArtifact(artifact in (Universal, packageZip), packageZip in Universal)
//--make sure the zip gets made before the publishing commands for the added artifacts
publish := (publish dependsOn (packageBin in Universal)).value
publishM2 := (publishM2 dependsOn (packageBin in Universal)).value
publishLocal := (publishLocal dependsOn (packageBin in Universal)).value
The key pieces of this solution came from a comment from yanns and dgrandes
EDIT4 Most of the text below the horizontal rule doesn't have anything to do with the real problem. At the beginning I thought forking is the problem, but that is not true.
I'm trying to run all the tests of an aggregates subprojects. Before all the tests in a subproject a setup method should run, and after the tests in a subproject a cleanup method should run.
When running the test task for the aggregate project, I expect the following sequence
setup method of project A is called
tests for project A are executed
cleanup method for project A is called
the same for project B
But the sequence is:
setup methods of project A and B are called
tests of project A are executed
tests of project B are executed
cleanup methods of project A and B are called
A build script having this behaviour can be found here.
How can I fix this problem to get my expected sequence?
I'm using forking to run the tests in my subprojects.
For every subproject a mongo db is started before the tests and stopped
after the tests.
The tests within one project run sequentially; this works well if I run the tests for a single project.
But if I run the task test for project root (the aggregate containing the sub projects),
I want the forked jvms to be started sequentially, i.e.
the jvm for project A is forked and its tests are executed
the jvm for project B is forked and its tests are executed
...
But it looks like the jvms are started in parallel; this is not what I want.
I tried the following (which should be set to 1 already, according to the documentation):
concurrentRestrictions in Test := Seq(
Tags.limit(Tags.ForkedTestGroup, 1)
)
But it didn't work. Directly after starting the test task, the following is
printed (before any test log is printed) from my setup method:
startupDb, thread name = pool-4-thread-5
startupDb, thread name = pool-4-thread-7
startupDb, thread name = pool-4-thread-2
startupDb, thread name = pool-4-thread-6
startupDb, thread name = pool-4-thread-8
startupDb, thread name = pool-4-thread-3
startupDb, thread name = pool-4-thread-9
These are my test related settings:
parallelExecution in Test := false,
testOptions in Test += Tests.Setup( () => MongoTest.startupDb() ),
testOptions in Test += Tests.Cleanup( () => MongoTest.shutdownDb() ),
fork in Test := true,
concurrentRestrictions in Test := Seq(
Tags.limit(Tags.ForkedTestGroup, 1)
)
It's important for me to use forking, but in the way as described above.
I'm using sbt 0.13.0 on Windows 7.
EDIT I created a gist with a sample build.
EDIT2 In the documentation it says:
Control the number of forked JVMs allowed to run at the same time by setting the limit on Tags.ForkedTestGroup tag, which is 1 by default
So this should work theoretically.
Is this a bug? If not, how can I manage to do that?
EDIT3 It looks like forking is not the problem. The problem is that the setup methods are called immediately for all subprojects.
Hmm. Trivial answer convert to comment. It sounds like you are asking the same question that got solved in the other question (see comments). The answer seems to be
parallelExecution in ThisBuild := false
and I found another blog post that suggested
parallelExecution in Global := false
to be the answer. The "Global" answer also suggested that would turn off parallel execution for things like compilation but that's probably less important than running tests.
As of sbt 0.12, you can make all test tasks in all subprojects run serially with this setting:
concurrentRestrictions in Global += Tags.limit(Tags.Test, 1)
(Tags.Test is a built in semantic tag for all test tasks).
See: Parallel-Execution in the SBT manual.
There is another way to prevent parallel execution. You can make the test tasks of the different projects depend on each other:
test in Project2 := (test in Project2).dependsOn(test in Project1).value
parallelExecution in Test in Project2 := false