I face the following problem when using SBT. If I add this line to build.sbt:
unmanagedResourceDirectories in Compile <+= baseDirectory( _ / "src/main/scala" )
Incremental compilation breaks in a very tricky and not-nice way. A full example on how to reproduce the bug is here: https://github.com/vn971/sbt-incremental-bug
It's basically 2 files. Implicits.scala:
object MyImplicits {
implicit def stringToInt(str: String) = 1
}
And Usage.scala:
import MyImplicits._
object MyUsage {
def a: Int = ""
}
Now, in order to reproduce the incremental compilation bug, you have to do consequent changes to these files:
comment out the method in Usage.scala. Save file, re-compile.
uncomment it, save and re-compile.
comment out the method in Implicits.scala. Save file, re-compile.
Since MyImplicits.stringToInt is undoubtedly used in Usage.scala, it should not compile. But it does, with incremental compilation.
Thoughts? Questions? If you need more details than already provided, check out the minimalistic project I've linked to.
Related
I have a few enumerations implemented as sealed traits and case objects. I prefer using the ADT approach because of the non-exhaustive warnings and mostly because we want to avoid the type erasure. Something like this
sealed abstract class Maker(val value: String) extends Product with Serializable {
override def toString = value
}
object Maker {
case object ChryslerMaker extends Vendor("Chrysler")
case object ToyotaMaker extends Vendor("Toyota")
case object NissanMaker extends Vendor("Nissan")
case object GMMaker extends Vendor("General Motors")
case object UnknownMaker extends Vendor("")
val tipos = List(ChryslerMaker, ToyotaMaker, NissanMaker,GMMaker, UnknownMaker)
private val fromStringMap: Map[String, Maker] = tipos.map(s => s.toString -> s).toMap
def apply(key: String): Option[Maker] = fromStringMap.get(key)
}
This is working well so far, now we are considering providing access to other programmers to our code to allow them to configure on site. I see two potential problems:
1) People messing up and writing things like:
case object ChryslerMaker extends Vendor("Nissan")
and people forgetting to update the tipos
I have been looking into using a configuration file (JSON or csv) to provide these values and read them as we do with plenty of other elements, but all the answers I have found rely on macros and seem to be extremely dependent on the scala version used (2.12 for us).
What I would like to find is:
1a) (Prefered) a way to create dynamically the case objects from a list of strings making sure the objects are named consistently with the value they hold
1b) (Acceptable) if this proves too hard a way to obtain the objects and the values during the test phase
2) Check that the number of elements in the list matches the number of case objects created.
I forgot to mention, I have looked briefly to enumeratum but I would prefer not to include additional libraries unless i really understand the pros and cons (and right now I am not sure how enumerated compares with the ADT approach, if you think this is the best way and can point me to such discussion that would work great)
Thanks !
One idea that comes to my mind is to create an SBT SourceGenerator task.
That will read an input JSON, CSV, XML or whatever file, that is part of your project and will create a scala file.
// ----- File: project/VendorsGenerator.scala -----
import sbt.Keys._
import sbt._
/**
* An SBT task that generates a managed source file with all Scalastyle inspections.
*/
object VendorsGenerator {
// For demonstration, I will use this plain List[String] to generate the code,
// you may change the code to read a file instead.
// Or maybe this will be good enough.
final val vendors: List[String] =
List(
"Chrysler",
"Toyota",
...
"Unknow"
)
val generatorTask = Def.task {
// Make the 'tipos' List, which contains all vendors.
val tipos =
vendors
.map(vendorName => s"${vendorName}Vendor")
.mkString("val tipos: List[Vendor] = List(", ",", ")")
// Make a case object for each vendor.
val vendorObjects = vendors.map { vendorName =>
s"""case object ${vendorName}Vendor extends Vendor { override final val value: String = "${vendorName}" }"""
}
// Fill the code template.
val code =
List(
List(
"package vendors",
"sealed trait Vendor extends Product with Serializable {",
"def value: String",
"override final def toString: String = value",
"}",
"object Vendors extends (String => Option[Vendor]) {"
),
vendorObjects,
List(
tipos,
"private final val fromStringMap: Map[String, Vendor] = tipos.map(v => v.toString -> v).toMap",
"override def apply(key: String): Option[Vendor] = fromStringMap.get(key.toLowerCase)",
"}"
)
).flatten
// Save the new file to the managed sources dir.
val vendorsFile = (sourceManaged in Compile).value / "vendors.scala"
IO.writeLines(vendorsFile, code)
Seq(vendorsFile)
}
}
Now, you can activate your source generator.
This task will be run each time, before the compile step.
// ----- File: build.sbt -----
sourceGenerators in Compile += VendorsGenerator.generatorTask.taskValue
Please note that I suggest this, because I have done it before and because I don't have any macros nor meta programming experience.
Also, note that this example relays a lot in Strings, which make the code a little bit hard to understand and maintain.
BTW, I haven't used enumeratum, but giving it a quick look looks like the best solution to this problem
Edit
I have my code ready to read a HOCON file and generate the matching code. My question now is where to place the scala file in the project directory and where will the files be generated. I am a little bit confused because there seems to be multiple steps 1) compile my scala generator, 2) run the generator, and 3) compile and build the project. Is this right?
Your generator is not part of your project code, but instead of your meta-project (I know that sounds confusing, you may read this for understanding that) - as such, you place the generator inside the project folder at the root level (the same folder where is the build.properties file for specifying the sbt version).
If your generator needs some dependencies (I'm sure it does for reading the HOCON) you place them in a build.sbt file inside that project folder.
If you plan to add unit test to the generator, you may create an entire scala project inside the meta-project (you may give a look to the project folder of a open source project (Yes, yes I know, confusing again) in which I work for reference) - My personal suggestion is that more than testing the generator itself, you should test the generated file instead, or better both.
The generated file will be automatically placed in the src_managed folder (which lives inside target and thus it is ignored from your source code version control).
The path inside that is just by order, as everything inside the src_managed folder is included by default when compiling.
val vendorsFile = (sourceManaged in Compile).value / "vendors.scala" // Path to the file to write.`
In order to access the values defined in the generated file on your source code, you only need to add a package to the generated file and import the values from that package in your code (as with any normal file).
You don't need to worry about anything related with compilation order, if you include your source generator in your build.sbt file, SBT will take care of everything automatically.
sourceGenerators in Compile += VendorsGenerator.generatorTask.taskValue // Activate the source generator.
SBT will re-run your generator everytime it needs to compile.
"BTW I get "not found: object sbt" on the imports".
If the project is inside the meta-project space, it will find the sbt package by default, don't worry about it.
I'm trying to run a code generator, and passing it the filename to write the output:
resourceGenerators in (proj, Compile) += Def.task {
val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
(runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value
Seq(file)
}.value
However, this gives me:
build.sbt:172: error: Illegal dynamic reference: file
(runMain in (proj, Compile)).toTask(s"api.swagger.SwaggerDump $file").value
Your code snippet has two problems:
You use { ... }.value instead of { ... }.taskValue. The type of resource generators is Seq[Task[Seq[File]]] and when you do value, you get Seq[File] not Task[Seq[File]]. That causes a legitimate compile error.
The dynamic variable file is used as the argument of toTask, which the current macro implementation prohibits.
Why static?
Sbt forces task implementations to have static dependencies on other tasks. Otherwise, sbt cannot perform task deduplication and cannot provide correct information in the inspect commands. That means that whichever task evaluation you perform inside a task cannot depend on a variable (a value known only at runtime), as your file in toTask does.
To overcome this limitation, there exists dynamic tasks, whose body allows you to return a task. Every "dynamic dependency" has to be defined inside a dynamic task, and then you can depend on the hoisted up dynamic values in the task that you return.
Dynamic solution
The following Scastie is the correct implementation of your task. I copy-paste the code so that folks can have a quick look, but go to that Scastie to check that it successfully compiles and runs.
resourceGenerators in (proj, Compile) += Def.taskDyn {
val file = (resourceManaged in (proj, Compile)).value / "swagger.yaml"
Def.task {
(runMain in (proj, Compile))
.toTask(s"api.swagger.SwaggerDump $file")
.value
Seq(file)
}
}.taskValue
Discussion
If you had fixed the taskValue error, should your task implementation correctly compile?
In my opinion, yes, but I haven't looked at the internal implementation good enough to assert that your task implementation does not hinder task deduplication and dependency extraction. If it does not, the illegal reference check should disappear.
This is a current limitation of sbt that I would like to get rid of, either by improving the whole macro implementation (hoisting up values and making sure that dependency analysis covers more cases) or by just improving the "illegal references checks" to not be over pessimistic. However, this is a hard problem, takes time and it's not likely to happen in the short term.
If this is an issue for you, please file a ticket in sbt/sbt. This is the only way to know the urgency of fixing this issue, if any. For now, the best we can do is to document it.
Is there any working solution for dropping into REPL console with for Scala 2.10?
This is mainly for debugging purpose - I want to pause in the middle of execution, and have a REPL console where I can inspect values and test the program's logic using complex expressions within my program at the current state of execution. Those who have programmed in Ruby might know similar function: the binding.pry.
AFAIK, Scala 2.9 and under used to have breakIf but it has been removed from the later versions. Using ILoop seems to be the new way but introduced issues due to sbt not adding scala-library to the classpath.
Several solutions such as this and this seem to offer a good workaround but my point is there must be a solution where I don't have to spend hours or even days just to make the REPL working.
In short, there's a lot more boilerplate steps involved - this is in contrast with binding.pry which is just a line of code with no additional boilerplate.
I am not aware if there's an issue introduced in executing the program as an sbt task as opposed to if running the program executable directly, but for development purpose I am currently running and testing my program using sbt task.
You could easily reimplement the breakIf method in your code. I don't think there is much cleaner way of doing that.
First you have to add a scala compiler library to your build.sbt
libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
Once that's done you can implement breakIf
import scala.reflect.ClassTag
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ILoop, NamedParam}
def breakIf[T](assertion: => Boolean, args: NamedParam*)(implicit tag: ClassTag[T]) = {
val repl = new ILoop()
repl.settings = new Settings()
repl.settings.embeddedDefaults[T]
repl.settings.Yreplsync.value = true
repl.in = repl.chooseReader(repl.settings)
repl.createInterpreter()
args.foreach(p => repl.bind(p.name, p.tpe, p.value))
repl.loop()
repl.closeInterpreter()
}
I think it's pretty straight forward, the only tricky part is that you have to set-up the classpath properly. You need to call embeddedDefaults with a class from your project (see my answer to another question).
You can use the new breakIf as follows:
val x = 10
breakIf[X](assertion = true, NamedParam("x", "Int", x))
Where X is just some of your classes.
I don't know if this answers your question, because it's hard to measure what is easy and what is hard.
Additionally just as a side note - if you want to use it for debugging purposes, why not use a debugger. I guess most of the debuggers can connect to a program, stop at a breakpoint and evaluate expressions in that context.
Edit
Seems like it doesn't work on current release of Scala 2.10, the working code seems to be:
import scala.reflect.ClassTag
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ILoop, NamedParam}
def breakIf[T](assertion: => Boolean, args: NamedParam*)(implicit tag: ClassTag[T]) = {
val repl = new ILoop() {
override protected def postInitialization(): Unit = {
addThunk(args.foreach(p => intp.bind(p)))
super.postInitialization()
}
}
val settings = new Settings()
settings.Yreplsync.value = true
settings.usejavacp.value = true
settings.embeddedDefaults[T]
args.foreach(repl.intp.rebind)
repl.process(settings)
}
and usage is like
val x = 10
breakIf[X](assertion = true, NamedParam("x", x))
I was looking at this recently and found Ammonite to be a sufficient solution for my needs.
Add Ammonite to your library dependencies:
libraryDependencies += "com.lihaoyi" % "ammonite" % "1.6.0" cross CrossVersion.full
Invoke Ammonite where you want to drop to the REPL shell:
ammonite.Main().run()
Note that you have to pass any variables you want bound inside of run, e.g. run("var1" -> var1). Have a look at their example - Instantiating Ammonite.
Assume you have two SBT projects, one called A and another called B
A has a subproject called macro, that follows the exact same pattern as showed here (http://www.scala-sbt.org/0.13.0/docs/Detailed-Topics/Macro-Projects.html). In other words, A has a subproject macro with a package that exposes a macro (lets called it macrotools). Now both projects, A and B, use the macrotools package (and A and B are strictly separate projects, B uses A via dependancies in SBT, with A using publish-local)
Now, A using A's macrotools package is fine, everything works correctly. However when B uses A macrotools package, the following error happens
java.lang.IllegalAccessError: tried to access method com.monetise.waitress.types.Married$.<init>()V from class com.monetise.waitress.types.RelationshipStatus$
For those wondering, the macro is this one https://stackoverflow.com/a/13672520/1519631, so in other words, this macro is what is inside the macrotools package
This is also related to my earlier question Macro dependancy appearing in POM/JAR, except that I am now using SBT 0.13, and I am following the altered guide for SBT 0.13
The code being referred to above is, in this case, this is what is in B, and A is com.monetise.incredients.macros.tools (which is a dependency specified in build.sbt)
package com.monetise.waitress.types
import com.monetise.ingredients.macros.tools.SealedContents
sealed abstract class RelationshipStatus(val id:Long, val formattedName:String)
case object Married extends RelationshipStatus(0,"Married")
case object Single extends RelationshipStatus(1,"Single")
object RelationshipStatus {
// val all:Set[RelationshipStatus] = Set(
// Married,Single
// )
val all:Set[RelationshipStatus] = SealedContents.values[RelationshipStatus]
}
As you can see, when I use whats commented, the code works fine (the job of the macro is to fill the Set with all the case objects in an ADT). When I use the macro version, i.e. SealedContents.values[RelationshipStatus] is when I hit the java.lang.IllegalAccessError
EDIT
Here are the repos containing the projects
https://github.com/mdedetrich/projectacontainingmacro
https://github.com/mdedetrich/projectb
Note that I had to do some changes, which I forgot about earlier. Because the other project needs to depend on the macro as well, the following 2 lines to disable macro publishing have been commented out
publish := {},
publishLocal := {}
In the build.scala. Also note this is a runtime, not a compile time error
EDIT 2
Created a github issue here https://github.com/sbt/sbt/issues/874
This issue is unrelated to SBT. It looks like the macro from Iteration over a sealed trait in Scala? that you're using has a bug. Follow the link to see a fix.
here is my test case , while i right click the file eclipse doest not show any run as junit test option. I try to manual create run configuration but does not take any sense.
scala version:2.8.1 scalatest:1.3 eclipse:3.6.2
package org.jilen.cache.segment
import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner
import org.scalatest.FlatSpec
import org.scalatest.matchers.ShouldMatchers
#RunWith(classOf[JUnitRunner])
class RandomSegmentSpec extends FlatSpec with ShouldMatchers {
val option = SegmentOptions()
"A Simple Segment" should "contains (douglas,lea) after put into" in {
val segment = RandomSegment.newSegment(option)
segment.put("douglas", "lea")
segment("douglas") should be("lea")
}
it should "return null after (douglas,lea) is remove" in {
val segment = RandomSegment.newSegment(option)
segment.put("douglas", "lea")
segment -= ("douglas")
segment("douglas") should equal(null)
}
it should "contains nothing after clear" in {
val segment = RandomSegment.newSegment(option)
segment.put("jilen", "zhang")
segment.put(10, "ten")
segment += ("douglas" -> "lea")
segment += ("20" -> 20)
segment.clear()
segment.isEmpty should be(true)
}
}
I've encountered this seemingly randomly, and I think I've finally figured out why.
Unfortunately the plugin doesn't yet change package declarations when you move files, nor the class names when you rename files. (Given you can put multiple classes in one file, the latter will likely never be done.) If you are used to the renamings being done automagically in Eclipse, like I am, you're bound to get caught on this.
So... check carefully the following:
the package declaration in your Scala file matches the Eclipse package name
the name of the test class in the Scala file matches the name of the Scala file
I just ran into this, fixed both, and now my test runs!
This is a known problem with the Eclipse IDE for Scala. I'm currently working on the plugin for this. Watch this space.
I found Scalatest to be very bad at integrating with Eclipse (running the tests from eclipse showed that it ran them - but they would not pass or fail, but simply show up as passive blank boxes).
For some reason I could NOT get it to work after 3 hours of trying things!
Finally I tried specs2 - and it worked (Scala 2.9, Junit4 and Eclipse 3.6)!
They have a great doc here:
http://etorreborre.github.com/specs2/guide/org.specs2.guide.Runners.html#Runners+guide
Since I don't care which testing framework to use, I will try Specs2 purely from the convenience point of view.