I have a scala project which I build using sbt 13.5.
Since I implemented a module using shapeless, whenever sbt tries to incrementally compile the project it fails in resolving the required implicits:
[...] could not find implicit value for parameter mapper [...]
[...] could not find implicit value for parameter folder [...]
and so on.
sbt clean and sbt compile solves the issue, but the project is fairly big and this is painfully slowing down the compilation times and my productivity in turn, as fresh build can take several minutes.
Any idea of what's going on here?
some extra info
So, after some more thinking I made some hypothesis. The issue occurs when using shapeless records, and by looking at the generated files, I think it could be an issue with the macro that generates the singleton type for each record key.
My module takes an HList of ColParser[T, K] declared as:
sealed trait ColParser[T, K] {
val columnL Witness.Aux[K]
}
So the compiler generates a Witness for each ColParser using a macro, and I'm afraid sbt loses track of the generated macros when deciding what to recompile, but this is just a sketchy hypothesis.
As a matter of fact, whenever I change something in the code that invokes my module (e.g. I add/remove a ColParser from the HList), I get the above error.
Forcing the recompilation of the module (by deleting the generated .class) fixes the issue.
Related
I want to use sttp library with guice(with scalaguice wrapper) in my app. But seems it is not so easy to correctly bind things like SttpBackend[Try, Nothing]
SttpBackend.scala
Try[_] and Try[AnyRef] show some other errors, but still have no idea how it should be done correctly
the error I got:
kinds of the type arguments (scala.util.Try) do not conform to the expected kinds of the type parameters (type T).
[error] scala.util.Try's type parameters do not match type T's expected parameters:
[error] class Try has one type parameter, but type T has none
[error] bind[SttpBackend[Try, Nothing]].toProvider[SttpBackendProvider]
[error] ` ^
SttpBackendProvider looks like:
def get: SttpBackend[Try, Nothing] = TryHttpURLConnectionBackend(opts)
complete example in scastie
interesting that version scalaguice 4.1.0 show this error, but latest 4.2.2 shows error inside it with converting Nothing to JavaType
I believe you hit two different bugs in the Scala-Guice one of which is not fixed yet (and probably even not submitted yet).
To describe those issues I need a fast intro into how Guice and Scala-Guice work. Essentially what Guice do is have a mapping from type onto the factory method for an object of that type. To support some advanced features types are mapped onto some internal "keys" representation and then for each "key" Guice builds a way to construct a corresponding object. Also it is important that generics in Java are implemented using type erasure. That's why when you write something like:
bind(classOf[SttpBackend[Try, Nothing]]).toProvider(classOf[SttpBackendProvider])
in raw-Guice, the "key" actually becomes something like "com.softwaremill.sttp.SttpBackend". Luckily Guice developers have thought about this issue with generics and introduced TypeLiteral[T] so you can convey the information about generics.
Scala type system is more reach than in Java and it has some better reflection support from the compiler. Scala-Guice exploits it to map Scala-types on those more detailed keys automatically. Unfortunately it doesn't always work perfectly.
The first issue is the result of the facts that the type SttpBackend is defined as
trait SttpBackend[R[_], -S]
so it uses it expects its first parameter to be a type constructor; and that originally Scala-Guice used the scala.reflect.Manifest infrastructure. AFAIU such higher-kind types are not representable as Manifest and this is what the error in your question really says.
Luckily Scala has added a new scala.reflect.runtime.universe.TypeTag infrastructure to tackle this issue in a better and more consistent way and the Scala-Guice migrated to its usage. That's why with the newer version of Scala-Guice the compiler error goes away. Unfortunately there is another bug in the Scala-Guice that makes the code fail in runtime and it is a lack of handling of the Nothing Scala type. You see, the Nothing type is a kind of fake one on the JVM. It is one of the things where the Scala type system is more reach than the Java one. There is no direct mapping for Nothing in the JVM world. Luckily there is no way to create any value of the type Nothing. Unfortunately you still can create a classOf[Nothing]. The Scala-to-JVM compiler handles it by using an artificial scala.runtime.Nothing$. It is not a part of the public API, it is implementation details of specifically Scala over JVM. Anyway this means that the Nothing type needs additional handling when converting into the Guice TypeLiteral and there is none. There is for Any the cousin of Nothing but not for Nothing (see the usage of the anyType in TypeConversions.scala).
So there are really two workarounds:
Use raw Java-based syntax for Guice instead of the nice Scala-Guice one:
bind(new TypeLiteral[SttpBackend[Try, Nothing]]() {})
.toInstance(sttpBackend) // or to whatever
See online demo based on your example.
Patch the TypeConversions.scala in the Scala-Guice as in:
private[scalaguice] object TypeConversions {
private val mirror = runtimeMirror(getClass.getClassLoader)
private val anyType = typeOf[Any]
private val nothingType = typeOf[Nothing] // added
...
def scalaTypeToJavaType(scalaType: ScalaType): JavaType = {
scalaType.dealias match {
case `anyType` => classOf[java.lang.Object]
case `nothingType` => classOf[scala.runtime.Nothing$] //added
...
I tried it locally and it seems to fix your example. I didn't do any extensive tests so it might have broken something else.
We have problems when using Scalac -Xfatal-warnings in the following cases:
Implicit vals used by macros internally
Internal vals that macros auto-generate
In both cases, we see Scalac failing to compile because it detects some values are not used, while we know they are (simply, when we remove them, the code doesn't compile anymore)
Although the two might be symptoms of the same problem in Scalac, they boil down to the same issue for us: we need to disable the -Ywarn-unused in Scala 2.11.12
Is there a way to exclude specific class files so they won't be affected by the compiler flag?
As far as I know there is no way of disabling scalac flag for just one file (if you compile your whole project at once by e.g sbt). You can extract class into separate module with different compile flags.
In case of implicit vals used internally in macros, personally I use -Ywarn-macros:after flag, which make these implicits used in macro count as used. (Talking about Scala 2.12.4).
Assume you have two SBT projects, one called A and another called B
A has a subproject called macro, that follows the exact same pattern as showed here (http://www.scala-sbt.org/0.13.0/docs/Detailed-Topics/Macro-Projects.html). In other words, A has a subproject macro with a package that exposes a macro (lets called it macrotools). Now both projects, A and B, use the macrotools package (and A and B are strictly separate projects, B uses A via dependancies in SBT, with A using publish-local)
Now, A using A's macrotools package is fine, everything works correctly. However when B uses A macrotools package, the following error happens
java.lang.IllegalAccessError: tried to access method com.monetise.waitress.types.Married$.<init>()V from class com.monetise.waitress.types.RelationshipStatus$
For those wondering, the macro is this one https://stackoverflow.com/a/13672520/1519631, so in other words, this macro is what is inside the macrotools package
This is also related to my earlier question Macro dependancy appearing in POM/JAR, except that I am now using SBT 0.13, and I am following the altered guide for SBT 0.13
The code being referred to above is, in this case, this is what is in B, and A is com.monetise.incredients.macros.tools (which is a dependency specified in build.sbt)
package com.monetise.waitress.types
import com.monetise.ingredients.macros.tools.SealedContents
sealed abstract class RelationshipStatus(val id:Long, val formattedName:String)
case object Married extends RelationshipStatus(0,"Married")
case object Single extends RelationshipStatus(1,"Single")
object RelationshipStatus {
// val all:Set[RelationshipStatus] = Set(
// Married,Single
// )
val all:Set[RelationshipStatus] = SealedContents.values[RelationshipStatus]
}
As you can see, when I use whats commented, the code works fine (the job of the macro is to fill the Set with all the case objects in an ADT). When I use the macro version, i.e. SealedContents.values[RelationshipStatus] is when I hit the java.lang.IllegalAccessError
EDIT
Here are the repos containing the projects
https://github.com/mdedetrich/projectacontainingmacro
https://github.com/mdedetrich/projectb
Note that I had to do some changes, which I forgot about earlier. Because the other project needs to depend on the macro as well, the following 2 lines to disable macro publishing have been commented out
publish := {},
publishLocal := {}
In the build.scala. Also note this is a runtime, not a compile time error
EDIT 2
Created a github issue here https://github.com/sbt/sbt/issues/874
This issue is unrelated to SBT. It looks like the macro from Iteration over a sealed trait in Scala? that you're using has a bug. Follow the link to see a fix.
So I'm playing with writing a battlecode player in Scala. In battlecode certain classes are disallowed and there is a runtime exception if you ever try to access them. When I use the Array.fill function I get a message from the battlecode server saying [java] Illegal class: scala/reflect/Manifest$. This is the offending line:
val g_score = Array.fill[Int](rc.getMapWidth(), rc.getMapHeight())(0)
The method takes an implicit ClassManifest argument which has the following documentation:
A ClassManifest[T] is an opaque descriptor for type T. It is used by the compiler
to preserve information necessary for instantiating Arrays in those cases where
the element type is unknown at compile time.
But I do know the type of the array elements at compile time, as shown above I explicitly state that they will be Int. Is there a way to avoid this? To workaround I've written my own version of Array.fill. This seems like a hack. As an aside, does Scala have real 2D arrays? Array.fill seems to return an Array[Array[T]] which is the only way I found to write my own. This also seems inelegant.
Edit: Using Scala 2.9.1
For background information, see this related question: What is a Manifest in Scala and when do you need it?. In this answer, you will find an explanation why manifests are needed for arrays.
In short: Although the JVM uses type erasure, arrays are an exception and need a manifest. Since you could compile your code, that manifest was found (manifests are always available for proper types). Your error occurs at runtime.
I don't know the details of the battlecode server, but there are two possibilities: Either you are running your compiled classes with a binary incompatible version of Scala (difference in major version, e.g. compiled with Scala 2.9 and server uses 2.10). Or the server doesn't even have the scala-library.jar on its class path.
As said in the comment, manifests are deprecated in Scala 2.10 and replaced by ClassTag.
EDIT: So it seems the class loader is artificially restricting the allowed classes. My suggestion would be: Add a helper Java class. You can easily mix Java and Scala code. If it's just about the Int-Array instantiation, you could provide something like:
public static class Helper {
public static int[][] makeArray(int d1, int d2) { return new int[d1][d2](); }
}
(hope that's valid java code, a bit rusty)
Also, have you tried to create the outer array with new Array[Array[Int]](d1), and then iterate to create the inner arrays?
Someday, I'd like to learn Scala. What I see about the language from people who like it is very encouraging.
Today, though, is not that day. Today, I'd just like to make some changes to my team's build file. Unfortunately, this build file was put together with SBT, and is nearly incomprehensible.
My main issue is that it appears to me that SBT introduces some huge collection of new operators that do things with strings and lists to create some sort of sbt object. For example, in sbt:
"args4j" % "args4j" % "2.0.12"
Apparently is actually defined; however, I can't even tell what type it is at the scala repl, since at the repl I get the sensible error:
scala> val tstcrap = "args4j" % "args4j" % "2.0.12"
<console>:6: error: value % is not a member of java.lang.String
val tstcrap = "args4j" % "args4j" % "2.0.12"
I get this error even after setting up the classpath to include the sbt-launch.jar file and doing import sbt._.
Likewise, I'm dealing with stuff like this:
val jarSources = (descendents(classesOutput ##, "*") ---
assemblyExclude(classesOutput ##))
What's that ## operator, what's that --- doing, and more importantly what is the type of this expression? Are all these new operators documented somewhere, and is there some way to get a scala repl that's using the same language as is used in the sbt build files?
Looking at this sbt file reminds me of trying to decipher perl without ever reading any of the relevant man pages. (Not a recommended activity)
Update: After looking at the links in the answer below, and looking at other questions and answers tagged sbt, I've come across the major piece of scala knowledge that I was missing: scala allows one to define implicit conversions that will be invoked before methods are resolved. In this case, sbt defines inside the ManagedProject trait, an implicit conversion from String to the private class sbt.GroupID, so that
"a" % "b"
Is really something like
(new GroupID("a")) % "b"
I imagine the resolution order and other rules around implicit conversions must get very complicated; it almost reminds me of the nightmares you can introduce in C++ with operator overloading when done through non-member functions.
Since an SBT build file is a full-fledged Scala source file and relies on some libraries provided by SBT itself, it's difficult to cover SBT well without relying on some familiarity with Scala. I'm not aware of such a guide.
For the specific questions you raise, I think these wiki pages will help:
% operator for strings: http://code.google.com/p/simple-build-tool/wiki/LibraryManagement
## and --- operators: http://code.google.com/p/simple-build-tool/wiki/Paths
If you want to get a Scala REPL running with the SBT libraries available, try this:
$ sbt console-project
Some other useful commands are listed at http://code.google.com/p/simple-build-tool/wiki/RunningSbt .
Update 2016 (5 years later).
This is not a complete guide, but the article "Sbt heiroglyphs and multi-projects explained" from Divan Visagie can help starting to use sbt.
Plus, the sbt documentation is quite complete nowadays, and covers multiple projects in a single build.
The '---' operator is described in the PathFinder (since the 0.2 version).