I am trying to setup SBT to compile an existing project which does not use the maven directory structure. I am using the full configuration and have set my javaSource & resourceDirectory settings as follows:
def settings = Defaults.defaultSettings ++ Seq(
resourceDirectory in Compile <<= baseDirectory( _ / "java" ),
javaSource in Compile <<= baseDirectory( _ / "java" )
)
Now I want to be able to filter the resources we include in the jar artifact, as we currently do with ant, plus exclude .java files as our resources are mixed in with source code. For example:
<fileset dir="java" includes="**/*.txt, **/*.csv" excludes="**/*.java" />
Is there any way to do this?
Use defaultExcludes scoped for the unmanagedResources task and optionally the configuration. For example, this setting excludes .java files from the main resources:
defaultExcludes in Compile in unmanagedResources := "*.java"
in Compile restricts this setting to only apply to main resources. By using in Test instead, it would apply only to test resources. By omitting a configuration (that is, no in Compile or in Test), the setting would apply to both main and test resources.
in unmanagedResources applies these excludes for resources only. To apply excludes to sources, for example, the scope would be in unmanagedSources. The reason for the unmanaged part is to emphasize that these apply to unmanaged (or manually edited) sources only.
The defaultExcludes key has type sbt.FileFilter, so the setting value must be of this type. In the example above, "*.java" is implicitly converted to a FileFilter. * is interpreted as a wildcard and so the filter accepts files with a name that ends in '.java'. To combine filters, you use || and &&. For example, if .scala files needed to be excluded as well, the argument to := would be:
"*.java" || "*.scala"
In the original Ant fileset, the include and exclude filters select mutually exclusive sets of files, so only one is necessary.
It is also possible to directly build the Seq[File] for unmanagedResources. For example:
unmanagedResources in Compile <<=
unmanagedResourceDirectories in Compile map { (dirs: Seq[File]) =>
( dirs ** ("*.txt" || "*.csv" -- "*.java") ).get
}
The ** method selects all descendents that match the FileFilter argument. You can verify that the files are selected as you expect by running show unmanaged-resources.
For sbt 1.48 I needed this style:
.settings(
Compile / sources := Seq(file("/path/to/your/file"))
)
Related
(Example project provided) I cannot get sbt-concat to work as designed to find and concatenate stylesheets that result from styles that may be produced from preprocessor tasks. In my production app, I'm trying to use it to bundle select minified output files from sbt-sass. It does not work within the complex setup of that project, so I created an example project to see if I could get it to work at all. It does not work in the example project either. Here is a test project build.sbt that tries to create several bundles, with just about every possibility I can think of, just to see if any of them work (public Github repo, which you should be able to clone and immediately replicate the problem):
import com.typesafe.sbt.web.Import.WebKeys._
import com.typesafe.sbt.web.pipeline.Pipeline
name := """sbt-concat-test"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala, SbtWeb)
scalaVersion := "2.11.1"
libraryDependencies ++= Seq(
jdbc,
anorm,
cache,
ws
)
resolvers += Resolver.sonatypeRepo("releases")
includeFilter in (Assets, LessKeys.less) := "*.less"
excludeFilter in (Assets, LessKeys.less) := "_*.less"
val myPipelineTask = taskKey[Pipeline.Stage]("Some pipeline task")
myPipelineTask := { mappings => println(mappings); mappings }
pipelineStages := Seq(myPipelineTask, concat)
Concat.groups := Seq(
"style-group1.css" -> group(sourceDirectory.value ** "*.css"),
"style-group2.css" -> group(baseDirectory.value ** "*.css"),
"style-group3.css" -> group((sourceDirectory in Assets).value ** "*.css"),
"style-group4.css" -> group(target.value ** "*.css"),
"style-group5.css" -> group(Seq("core.css", "styles/core.css", "assets/styles/core.css", "app/assets/styles/core.css")),
"style-group6.css" -> group(Seq("lessStyle.css", "ui/lessStyle.css", "styles/ui/lessStyle.css", "assets/styles/ui/lessStyle.css", "app/assets/styles/ui/lessStyle.css")),
"style-group7.css" -> group(Seq("sassStyle.css", "ui/sassStyle.css", "styles/ui/sassStyle.css", "assets/styles/ui/sassStyle.css", "app/assets/styles/ui/sassStyle.css")),
"style-group8.css" -> group(Seq("**/*.css"))
)
I run ; clean; reload; stage from activator to test. I see asset source files copied over into the target folder, with the following results for the declared bundles:
style-group1.css does not exist
style-group2.css contains the contents of button.css and core.css
style-group3.css contains the contents of core.css and button.css
style-group4.css does not exist
style-group5.css contains only the contents of core.css
style-group6.css contains only the contents of compiled lessStyle.scss
style-group7.css contains only the contents of compiled sassStyle.scss
style-group8.css does not exist
I do not understand why the 2nd and 3rd cases do not pick up the preprocessor-produced css files, yet the tailor-made 6th and 7th cases do. Perhaps notably, the result of myPipelineTask shows PathMappings for all source files, as well as the derived css and sourcemaps from the Sass and Less tasks.
According to Typesafe support, the source of my woes is the fact that sbt-concat implements its PathFinder logic in such a way as to only pick up assets that are verbatim the same filename as in the source directory. The sequence of relative filenames works for files in the target directory, but has no pattern matching. This is rather unfortunate.
What does work is to construct a Seq of output files that will exist post-compilation by using a PathFinder on the source directory. So for .scss files, something like:
Concat.groups := {
// Determine the output names of the style files to bundle
// This is really roundabout because sbt-concat only offers 2 ways of
// specifying files, relative paths and using a PathFinder, and the
// latter approach restricts itself to source files instead of output files.
val sourceDir = (sourceDirectory in Assets).value
val scssFiles = (sourceDir ** "*.scss").getPaths
val scssRelativePaths = scssFiles.map(_.stripPrefix(sourceDir.getPath).stripPrefix("/"))
val outputRelativePaths = scssRelativePaths.map(_.stripSuffix(".scss") + ".min.css")
Seq("bundle.min.css" -> group(outputRelativePaths))
}
As a side note, another quirk of sbt-concat is that it doesn't put its new files in its own directory in web-assets:assetsTarget to separate them from artifacts of other pipeline stages. The Concat.parentDir is also unnecessary because you can simply anything you would place in that variable as a prefix for your bundle file name directly.
I've been writing an SBT plugin that generates resources into resource_managed. I'm now looking to include these generated resources in the generated jar as the SBT docs detail:
Generating resources:
By default, generated resources are not included in the packaged source artifact. To do so, add them as you would other mappings. See Adding files to a package
I've read the docs but honestly how to do this I can't figure out. Can anyone explain it or point me to another project that does this so I can see how they do it?
First just to clarify, they are included in jars containing compiled classes. They are not included in jars containing sources.
By default, generated resources are not included in the packaged
source artifact.
For packageBin the generated files should already be included - just make sure you return all generated files from the generator method. Assuming you want to package them in the sources artifact, this is what you have to do.
Let's assume you have a generator that generates a property file.
lazy val generatePropertiesTask = Def.task {
val file = (Compile / resourceManaged).value / "stack-overflow" / "res.properties"
val contents = s"name=${name.value}\nversion=${version.value}"
IO.write(file, contents)
Seq(file)
}
resourceGenerators in Compile += generatePropertiesTask.taskValue
To include that in the generated sources you have to tell sbt to where the res.properties must be copied in the generated sources artefact. The task, which generates the packaged sources is called packageSrc, therefore you have to set mappings scoped to that task.
mappings in (Compile, packageSrc) += {
((resourceManaged in Compile).value / "stack-overflow" / "res.properties") -> "path/in/jar/res.properties"
}
Because your generator can generate many tasks, and mapping each by hand would be a tedious task, sbt gives you an utility to map multiple paths at once.
mappings in (Compile, packageSrc) ++= {
val allGeneratedFiles = ((resourceManaged in Compile).value ** "*") filter { _.isFile }
allGeneratedFiles.get pair relativeTo((resourceManaged in Compile).value)
}
The first line finds all generated files using path finders and second line maps them to their path in the target jar.
In one of my sub projects, I am trying to exclude *.conf and *.groovy files from my list of unmanaged resources:
excludeFilter in Compile in unmanagedResources := "*.conf" || "*.groovy"
Now, this works but has the unintended effect of removing the *.conf files from Test.
I tried to add the following includeFilter setting :
includeFilter in Test in unmanagedResources := "*.conf"
However, this does not work.
I figure that there is a relationship between Test and Compile that may be causing this issue.
Any suggestions would be helpful.
Thanks.
There are two issues here and you've identified the main one, which is the relationship between Test and Compile. The other is that a file must both be included by includeFilter and not be excluded by excludeFilter.
Test gets its settings from Compile if none are specified for Test explicitly. When you define excludeFilter in Compile, it applies to Test as well. So, you can define excludeFilter in Test to be the default, which is to ignore hidden files:
excludeFilter in Test := HiddenFileFilter
(Or, you can use NoFilter to not have any exclusions.)
Suppose I have a Scala program that creates files ending in .foo.
I'm building with sbt and want to remove these files whenever sbt clean is called.
Add additional directory to clean task in SBT build shows that a singe file can be added by
cleanFiles <+= baseDirectory { _ / "test.foo" }
However, it's unclear how to extend this to do:
cleanFiles <append> <*.foo>
All .foo files will be in the same directory, so I don't need to recursively check directories.
Though, that would also be interesting to see.
How can I configure sbt to clean files matching a wildcard, or regex?
Is it a bad design decision to have sbt clean remove files my program generates?
Should I instead use a flag in my program? Using sbt clean seems cleaner to me rather
than having to call sbt clean then sbt "run --clean".
This will find anything that matches *.foo in the base directory (but not child directories):
cleanFiles <++= baseDirectory (_ * "*.foo" get)
This works because Seq[File] gets implicitly converted to a PathFinder, which has methods such as * (match the pattern in the base directory) and ** (match the pattern including child directories). Then get converts it back to a Seq[File].
I am converting a legacy jar project to SBT and for strange reasons that are not easily solved, this project comes with "javax/servlet/Servlet.class" inside it. So I need to somehow exclude this class from the jar file generated by package-bin. How do I accomplish this ?. Preferably I would like to exclude using a wildcard (i.e. javax.*).
The SBT assembly plugin does look like it has features that will do this, but I am worried that relying on sbt assembly means that my jar project will not work in a muliti module project (i.e. if I include it as a dependency in a war file then the war projects needs to be told to run assembly on the dependent jar project rather than package-bin - but I may be mistaken here).
Each task declares the other tasks and settings that it uses. You can use inspect to determine these inputs as described on Inspecting Settings and in a recent tutorial-style blog post by John Cheng.
In this case, the relevant task used by packageBin is mappings. The mappings task collects the files to be included in the jar and maps them to the path in the jar. Some background is explained on Mapping Files, but the result is that mappings produces a value of type Seq[(File, String)]. Here, the File is the input file providing the content and the String is the path in the jar.
So, to modify the mappings for the packageBin task, filter out the paths from the default mappings that you don't want to include:
mappings in (Compile,packageBin) ~= { (ms: Seq[(File, String)]) =>
ms filter { case (file, toPath) =>
toPath != "javax/servlet/Servlet.class"
}
}
mappings in (Compile,packageBin) selects the mappings for the main package task (as opposed to test sources or the packageSrc task).
x ~= f means "set x to the result of applying function f to the previous value of x". (See More About Settings for details.)
The filter drops all pairs where the path corresponds to the Servlet class.
I came up with this solution, it defines a new compile task which depends on the previous compile task (thus effectively allowing me to hook in right after the source is compiled and before it's packaged)
def mySettings = {
// add functionality to the standard compile task
inConfig(Compile)(Seq(compile in Compile <<= (target,streams,compile in Compile) map{
(targetDirectory, taskStream, analysis) =>
import taskStream.log
// this runs after compile but before package-bin
recursiveListFiles(targetDirectory, ".*javax.*".r) foreach {
file =>
log.warn("deleting matched resource: " + file.getAbsolutePath())
IO.delete(file)
}
analysis
})) ++
Seq(name := "MyProject", version := "1.0", exportJars := true)
}
def recursiveListFiles(f: File, r: Regex): Array[File] = {
val these = f.listFiles
val good = these.filter(f => r.findFirstIn(f.getName).isDefined)
good ++ these.filter(_.isDirectory).flatMap(recursiveListFiles(_, r))
}
Its a little bit more complicated than what I had hoped but it allows me to do all sorts of modifications prior to packaging (in this case searching the target folder deleting all class files that matches a regular expression). Also it accomplished my second goal of sticking with the default SBT lifecycle.