How to avoid re-compiling generated source code - scala

Question

Generating boilerplate source code with sbt works fine:
sourceGenerators in Compile <+= sourceManaged in Compile map { srcDir =>
DslBoilerplate.generate(srcDir, Seq(
"path/to/a/definition/file"
))
}
When I run sbt compile this also compiles the generated source code files, thus producing some class files. I just don't want the generated source code to be re-compiled every time I re-compile the project during development.
So, from the class files I made a jar file and used this instead of the generated source/class files (I deleted those). This worked fine, now having access to the generated code through the jar file. Is there a way though to let sbt do the 4 steps (if needed?) in the initial project build?:
generate source code files
compile those files
create a jar from the produced class files
delete source and class files
(In this question they use the sbt.IO.jar method to create a jar but there they already have existing files...)
Or is there another better approach than making a jar to avoid re-compiling generated source code?
Update 1 (see update 2 below)
Thanks, Seth, for your answer! It worked great to avoid generating the source files with each project compilation since the cache now remembers that they have been created. I'll certainly use this feature, thanks!
But this was actually not what I had in mind with my original question. Sorry for not being clear enough. It might be clearer if we think of this as 2 transformations happening:
Input file ---1---> Source file (*.scala) ---2---> Target file (*.class)
where the transformations are
generation of source code (from some information in an input file) and
compilation of the generated source code
This all works fine when I compile the project with sbt compile.
But then if I "rebuild the project" (in IntelliJ), the generated source code (from the sbt compilation) will compile again, and that's what I want to avoid - but at the same time have access to that code. Is there any other way to avoid compilation than placing this code in a jar and then delete the source and target files?
So I tried to continue along that line of thought wrestling with sbt to make it create a source and target jar - still can't make the target jar. This is what I came up with so far (with help from this):
sourceGenerators in Compile += Def.task[Seq[File]] {
val srcDir = (sourceManaged in Compile).value
val targetDir = (classDirectory in Compile).value
// Picking up inputs for source generation
val inputDirs = Seq("examples/src/main/scala/molecule/examples/seattle")
// generate source files
val srcFiles = DslBoilerplate.generate(srcDir, inputDirs)
// prepare data to create jars
val srcFilesData = files2TupleRec("", srcDir)
val targetFilesData = files2TupleRec("", targetDir)
// debug
println("### srcDir: " + srcDir)
println("### srcFilesData: \n" + srcFilesData.mkString("\n"))
println("### targetDir: " + targetDir)
println("### targetFilesData: \n" + targetFilesData.mkString("\n"))
// Create jar from generated source files - works fine
val srcJar = new File("lib/srcFiles.jar/")
println("### sourceJar: " + srcJar)
sbt.IO.jar(srcFilesData, srcJar, new java.util.jar.Manifest)
// Create jar from target files compiled from generated source files
// Oops - those haven't been created yet, so this jar becomes empty... :-(
// Could we use dependsOn to have the source files compiled first?
val targetJar = new File("lib/targetFiles.jar/")
println("### targetJar: " + targetJar)
sbt.IO.jar(targetFilesData, targetJar, new java.util.jar.Manifest)
val cache = FileFunction.cached(
streams.value.cacheDirectory / "filesCache",
inStyle = FilesInfo.hash,
outStyle = FilesInfo.hash
) {
in: Set[File] => srcFiles.toSet
}
cache(srcFiles.toSet).toSeq
}.taskValue
def files2TupleRec(pathPrefix: String, dir: File): Seq[Tuple2[File, String]] = {
sbt.IO.listFiles(dir) flatMap {
f => {
if (f.isFile && f.name.endsWith(".scala")) Seq((f, s"${pathPrefix}${f.getName}"))
else files2TupleRec(s"${pathPrefix}${f.getName}/", f)
}
}
}
Maybe I still don't need to create jars? Maybe they shouldn't be created in the source generation task? I need help...
Update 2
Silly me!!! No wonder I can't make a jar with class files if I filter them with f.name.endsWith(".scala"), dohh
Since my initial question was not that clear, and Seth's answer is addressing an obvious interpretation, I'll accept his answer (after investigating more, I see that I should probably ask another question).

Answer 1

You want to use FileFunction.cached so that the source files aren't regenerated every time.
Here's an example from my own build:
Compile / sourceGenerators += Def.task[Seq[File]] {
val src = (Compile / sourceManaged).value
val base = baseDirectory.value
val s = streams.value
val cache =
FileFunction.cached(s.cacheDirectory / "lexers", inStyle = FilesInfo.hash, outStyle = FilesInfo.hash) {
in: Set[File] =>
Set(flex(s.log.info(_), base, src, "ImportLexer"),
flex(s.log.info(_), base, src, "TokenLexer"))
}
cache(Set(base / "project" / "flex" / "warning.txt",
base / "project" / "flex" / "ImportLexer.flex",
base / "project" / "flex" / "TokenLexer.flex")).toSeq
}.taskValue
Here the .txt and .flex files are input files to the generator. The actual work of generating the source files is farmed out to my flex method, which returns a java.io.File:
def flex(log: String => Unit, base: File, dir: File, kind: String): File =
...
You should be able to adapt this technique to your build.
FileFunction.cached is described in the API doc and in the sbt FAQ under "How can a task avoid redoing work if the input files are unchanged?" (http://www.scala-sbt.org/0.13/docs/Faq.html). (It would be nice if the material on caching was referenced from http://www.scala-sbt.org/0.13/docs/Howto-Generating-Files.html as well; currently it isn't.)