Given some method
def f[A,B](p: A)(implicit a: X[A,B], b: Y[B])
Does the order of a before b within the implicit parameter list matter for type inference?
I thought only the placement of parameters within different parameter lists matters, e.g. type information flows only through parameter lists from left to right.
I'm asking because I noticed that changing the order of implicit parameters within the singly implicit list made a program of mine compile.
Real example
The following code is using:
shapeless 2.1.0
Scala 2.11.5
Here is a simple sbt build file to help along with compiling the examples:
scalaVersion := "2.11.5"
libraryDependencies += "com.chuusai" %% "shapeless" % "2.1.0"
scalaSource in Compile := baseDirectory.value
Onto the example. This code compiles:
import shapeless._
import shapeless.ops.hlist.Comapped
class Foo {
trait NN
trait Node[X] extends NN
object Computation {
def foo[LN <: HList, N <: HList, TupN <: Product, FunDT]
(dependencies: TupN)
(computation: FunDT)
(implicit tupToHlist: Generic.Aux[TupN, LN], unwrap: Comapped.Aux[LN, Node, N]) = ???
// (implicit unwrap: Comapped.Aux[LN, Node, N], tupToHlist: Generic.Aux[TupN, LN]) = ???
val ni: Node[Int] = ???
val ns: Node[String] = ???
val x = foo((ni,ns))((i: Int, s: String) => s + i.toString)
}
}
and this code fails
import shapeless._
import shapeless.ops.hlist.Comapped
class Foo {
trait NN
trait Node[X] extends NN
object Computation {
def foo[LN <: HList, N <: HList, TupN <: Product, FunDT]
(dependencies: TupN)
(computation: FunDT)
// (implicit tupToHlist: Generic.Aux[TupN, LN], unwrap: Comapped.Aux[LN, Node, N]) = ???
(implicit unwrap: Comapped.Aux[LN, Node, N], tupToHlist: Generic.Aux[TupN, LN]) = ???
val ni: Node[Int] = ???
val ns: Node[String] = ???
val x = foo((ni,ns))((i: Int, s: String) => s + i.toString)
}
}
with the following compile error
Error:(22, 25) ambiguous implicit values:
both method hnilComapped in object Comapped of type [F[_]]=> shapeless.ops.hlist.Comapped.Aux[shapeless.HNil,F,shapeless.HNil]
and method hlistComapped in object Comapped of type [H, T <: shapeless.HList, F[_]](implicit mt: shapeless.ops.hlist.Comapped[T,F])shapeless.ops.hlist.Comapped.Aux[shapeless.::[F[H],T],F,shapeless.::[H,mt.Out]]
match expected type shapeless.ops.hlist.Comapped.Aux[LN,Foo.this.Node,N]
val x = foo((ni,ns))((i: Int, s: String) => s + i.toString)
^
Error:(22, 25) could not find implicit value for parameter unwrap: shapeless.ops.hlist.Comapped.Aux[LN,Foo.this.Node,N]
val x = foo((ni,ns))((i: Int, s: String) => s + i.toString)
^
Error:(22, 25) not enough arguments for method foo: (implicit unwrap: shapeless.ops.hlist.Comapped.Aux[LN,Foo.this.Node,N], implicit tupToHlist: shapeless.Generic.Aux[(Foo.this.Node[Int], Foo.this.Node[String]),LN])Nothing.
Unspecified value parameters unwrap, tupToHlist.
val x = foo((ni,ns))((i: Int, s: String) => s + i.toString)
^
Normally it should not matter. If you look at the language spec it makes no mention about resolution being dependent on parameter order.
I looked at the source code of shapeless, and I could not come up with any reason why this error would present itself.
And doing a quick search through the bug repo of the language I found a similar issue that was apparently resolved. But it does not state if the fix involved treating the symptom (making context bounds not break compilation) or the cause (restrictions on implicit parameter ordering.)
Therefore I would argue that this is a compiler bug, and it is tightly related to the issue linked in point 3.
Also, I would suggest you submit a bug report if you can find a second opinion that resulted from a more rigorous analysis than my own :)
Hope this puts your mind at rest. Cheers!
According to my reading of the comments of the issue mentioned by Lorand Szakacs, I come to the conclusion that the order of implicit parameters matters in the current version 2.11 of the Scala compiler.
This is because the developers participating in the discussion appear to assume that the order matters; they do not state it explicitly.
I'm not aware of the language spec mentioning anything about this topic.
Reordering them will only break code that explicitly passes them, as well as all compiled code. Everything else will be unaffected.
Related
(note: this problem is fixed as of Scala 2.13, see here: https://github.com/scala/scala/pull/6050)
I am working on a system of Scala types that involves chained implicits. This system behaves as I expected in many cases, but fails via diverging expansion in others. So far I haven't come up with a good explanation for the divergence, and I'm hoping the community can explain it for me!
Here is a simplified system of types that reproduces the problem:
object repro {
import scala.reflect.runtime.universe._
trait +[L, R]
case class Atomic[V](val name: String)
object Atomic {
def apply[V](implicit vtt: TypeTag[V]): Atomic[V] = Atomic[V](vtt.tpe.typeSymbol.name.toString)
}
case class Assign[V, X](val name: String)
object Assign {
def apply[V, X](implicit vtt: TypeTag[V]): Assign[V, X] = Assign[V, X](vtt.tpe.typeSymbol.name.toString)
}
trait AsString[X] {
def str: String
}
object AsString {
implicit def atomic[V](implicit a: Atomic[V]): AsString[V] =
new AsString[V] { val str = a.name }
implicit def assign[V, X](implicit a: Assign[V, X], asx: AsString[X]): AsString[V] =
new AsString[V] { val str = asx.str }
implicit def plus[L, R](implicit asl: AsString[L], asr: AsString[R]): AsString[+[L, R]] =
new AsString[+[L, R]] { val str = s"(${asl.str}) + (${asr.str})" }
}
trait X
implicit val declareX = Atomic[X]
trait Y
implicit val declareY = Atomic[Y]
trait Z
implicit val declareZ = Atomic[Z]
trait Q
implicit val declareQ = Assign[Q, (X + Y) + Z]
trait R
implicit val declareR = Assign[R, Q + Z]
}
Following is a demo of the behavior, with some working cases and then the diverging failure:
scala> :load /home/eje/divergence-repro.scala
Loading /home/eje/divergence-repro.scala...
defined module repro
scala> import repro._
import repro._
scala> implicitly[AsString[X]].str
res0: String = X
scala> implicitly[AsString[X + Y]].str
res1: String = (X) + (Y)
scala> implicitly[AsString[Q]].str
res2: String = ((X) + (Y)) + (Z)
scala> implicitly[AsString[R]].str
<console>:12: error: diverging implicit expansion for type repro.AsString[repro.R]
starting with method assign in object AsString
implicitly[AsString[R]].str
You'll be surprised to know you haven't done anything wrong! Well on a logical level at least. What you've encountered as error here is a known behavior of the Scala compiler when resolving implicits for recursive data structures. a good explanation of this behavior is given in the book The Type Astronaut's Guide to Shapeless:
Implicit resolution is a search process. The compiler uses heuristics to determine whether it is “converging” on a solution. If the heuristics don’t yield
favorable results for a particular branch of search, the compiler assumes the
branch is not converging and moves onto another.
One heuristic is specifically designed to avoid infinite loops. If the compiler
sees the same target type twice in a particular branch of search, it gives up
and moves on. We can see this happening if we look at the expansion for
CsvEncoder[Tree[Int]] The implicit resolution process goes through the
following types:
CsvEncoder[Tree[Int]] // 1
CsvEncoder[Branch[Int] :+: Leaf[Int] :+: CNil] // 2
CsvEncoder[Branch[Int]] // 3
CsvEncoder[Tree[Int] :: Tree[Int] :: HNil] // 4
CsvEncoder[Tree[Int]] // 5 uh oh
We see Tree[A] twice in lines 1 and 5, so the compiler moves onto
another branch of search. The eventual consequence is that it fails to
find a suitable implicit.
In your case if the compiler had kept going and not given up on so early it would have eventually reached the solution! But remember not every diverging implicit error is false compiler alarm. some are in fact diverging / infinitely expanding.
I know of two solutions to this issue:
macro based lazy evaluation of recursive types:
The shapeless library has a Lazy type that differs evaluation of it's Hlists head to runtime and as a result prevents this diverging implicit error. I find explaining or providing examples of it is beyond the OP topic. but you should check it.
create implicit checkpoints so that the implicit for the recursive type is available to the compiler beforehand:
implicitly[AsString[X]].str
implicitly[AsString[X + Y]].str
val asQ = implicitly[AsString[Q]]
asQ.str
{
implicit val asQImplicitCheckpoint: AsString[Q] = asQ
implicitly[AsString[R]].str
}
It's not a shame if you are a fan of neither of these solutions. the shapeless's Lazy solution while tried and true is still a third party library dependency and also with the removal of macros in scala 3.0 I'm not certain what'll become of all these macro based technics.
While this problem was caught in code using shapeless and kind-projector, this behavior could be reproduced without them.
Suppose I have simple typeclass for reifying typeclass instances with incomplete implementation (mirror of LiftAll).
sealed trait LiftAll1[F[_], In] {
type Out
def instances: Out
}
object LiftAll1 {
type Aux[F[_], In0, Out0] = LiftAll1[F, In0] {type Out = Out0}
implicit def unit[F[_]]: Aux[F, Unit, Unit] = new LiftAll1[F, Unit] {
type Out = Unit
def instances = Unit
}
}
And some very simple type class to test it
sealed class FirstIs[M, T]
object FirstIs {
implicit def firstIs[M, D]: FirstIs[M, (M, D)] = new FirstIs
}
Things are ok if I'll try to apply FirstIs partially via alias, and get instance via LiftAll1
type FirstIsInt[D] = FirstIs[Int, D]
implicitly[LiftAll1[FirstIsInt, Unit]]
But inlined partial type application leads to compilation error
implicitly[LiftAll1[({type lambda[x] = FirstIs[Int, x]})#lambda, Unit]]
//Error: could not find implicit value for parameter e: LiftAll1[[x]FirstIs[Int,x],Unit]
How partially applied typeclasses could be found in such situations?
As #Reactormonk suggested, the compiler was brought to its senses with following line in the build.sbt
scalacOptions += "-Ypartial-unification"
After that original piece of code, which is close to
import shapeless._, ops.hlist._
LiftAll[FirstIs[Int, ?], HNil]
was succesfully compiled
AFAIK problem was in scala compiler incapability to understand FirstIs[Int,_] as F[_]in supplied implicit def without direct type alias . Fortunately this was fixed in latest scala implementations.
In scala, the following code compiles properly:
class a {}
class b {}
object Main {
implicit class Conv[f, t](val v: f ⇒ t) extends AnyVal {
def conv = v
}
def main(args: Array[String]) {
val m = (a: a) ⇒ new b
m.conv
}
}
But for some reason the following fails to compile:
class a {}
class b {}
object Main {
type V[f, t] = f ⇒ t
implicit class Conv[f, t](val v: V[f, t]) extends AnyVal {
def conv = v
}
def main(args: Array[String]) {
val m = (a: a) ⇒ new b
m.conv
}
}
with the following message:
value conv is not a member of a => b
m.conv
Why does this happen?
EDIT: Yes, there is still an error even with
val m: V[a,b] = new V[a,b] { def apply(a: a) = new b }
In your first example, val v: f => t is inferred to a type signature [-A, +B], because it is shorthand for a Function of one parameter. Function1 has the type signature, Function1[-A, +B]. So, a type which is Contravariant in the A parameter and Covariant in the B parameter.
Then the lambda function, (a: a) => new b later in the code, has it's type inferred as a function from a to b. So, the type signature is identical and implicit resolution works.
In your second example type V[f, t] = f => t and the parameter created from it: val v: V[f, t], have their type explicitly specified as V[f, t]. The function f => t would still be [-A, +B], but you explicitly restrict your types to being Invariant in the type signature, so the type V is Invariant in both type parameters.
Later, when you declare: val m = (a: a) => new b the type signature of this would still be [-A, +B] as in the first example. Implicit resolution fails to work because the val is Contravariant in its first type parameter but the type V is Invariant in its first type parameter.
Changing the type signature of V to either V[-f, +t] or V[-f, t] resolves this and implicit resolution works once again.
This does raise a question about why the Covariance of the second type parameter is not a problem for implicit resolution, whilst the Contravariance of the first type parameter is. I played around and did a little bit of research. I came across a few interesting links, which indicates that there are definitely some limitations/issues around implicit resolution specifically when it comes to Contravariance.
A Scala Ticket related to Contravariance and implicit resolution.
A long discussion on google groups related to the ticket
Some workaround code for Contravariance and implicit resolutions in some scenarios.
I would have to defer to someone with knowledge of the internals of the Scala compiler and the resolution mechanics for more detail, but it seems that this is running afoul of some limitations around implicit resolution in the context of Contravariance.
For your third example, I think you mean:
class a {}
class b {}
object Main {
type V[f, t] = f => t
implicit class Conv[f, t](val v: V[f, t]) extends AnyVal {
def conv = v
}
def main(args: Array[String]) {
val m: V[a,b] = new V[a,b] { def apply(a: a) = new b }
m.conv // does not compile
}
}
This is an interesting one and I think it is a slightly different cause. Type Aliases are restricted in what they can change when it comes to variance, but making the variance stricter is allowed. I can't say for sure what is happening here, but here is an interesting Stack Overflow question related to type aliases and variance.
Given the complexity of implicit resolution, combined with the additional factors of the type declaration variance versus the implied Function1 variance I suspect the compiler is just not able to resolve anything in that specific scenario.
Changing to:
implicit class Conv(val v: V[_, _]) extends AnyVal {
def conv = v
}
means that it works in all scenarios, because you are basically saying to the compiler, that for the purposes of the Conv implicit class, you don't care about the variance of the type parameters on V.
e.g: the following also works
class a {}
class b {}
object Main {
type V[f, t] = f ⇒ t
implicit class Conv(val v: V[_, _]) extends AnyVal {
def conv = v
}
def main(args: Array[String]) {
val m = (a: a) ⇒ new b
m.conv
}
}
I just recently started learning scala and today I decided I wanted to write a CSV parser that would load nicely into case classes but store the data in rows (lists) of Shapeless's HList object so that I could get some exposure to type-level programming.
Here's what I have so far:
// LoadsCsv.scala
import shapeless._
import scala.collection.mutable
trait LoadsCsv[A, T <: HList] {
val rows: mutable.MutableList[T] = new mutable.MutableList[T]
def convert(t: T)(implicit gen: Generic.Aux[A, T]): A = gen.from(t)
def get(index: Int): A = {
convert(rows(index))
}
def load(file: String): Unit = {
val lines = io.Source.fromFile(file).getLines()
lines.foreach(line => rows += parse(line.split(",")))
}
def parse(line: Array[String]): T
}
And the object that's loading the data set:
// TennisData.scala
import shapeless._
case class TennisData(weather:String, low:Int, high:Int, windy:Boolean, play:Boolean)
object TennisData extends LoadsCsv[TennisData, String :: Int :: Int :: Boolean :: Boolean :: HNil] {
load("tennis.csv")
override def parse(line: Array[String]) = {
line(0) :: line(1).toInt :: line(2).toInt :: line(3).toBoolean :: line(4).toBoolean :: HNil
}
}
Things seem to be working alright until I added the get() with the conversion from the HList to the case class where I now get a compilation error. Why isn't the implicit getting loaded and what can I do to fix it or otherwise convert from the HList to the case class?
Error:(14, 17) could not find implicit value for parameter gen: shapeless.Generic.Aux[A,T]
return convert(rows(index))
^
I've been reading the shapeless documentation and it mentions that this area had been in flux between version 1 and 2, but I believe things should be working on my version of shapeless and scala so I suspect I've just done something incorrectly.
https://github.com/milessabin/shapeless/wiki/Migration-guide:-shapeless-1.2.4-to-2.0.0#iso-is-now-generic
For reference, I'm running scala 2.11.6 and shapeless 2.2.2
You're very close. The problem is that Scala isn't going to propagate implicit requirements up the call chain automatically for you. If you need a Generic[A, T] instance to call convert, then you'll have to make sure that one's in scope every time you call convert convert. If A and T are fixed (and are actually an case class-HList pair), Shapeless will generate one for you. In your get method, however, the compiler still knows nothing about A and T except that T is an HList, so you need to require the instance again in order to call convert:
def get(index: Int)(implicit gen: Generic.Aux[A, T]): A = convert(rows(index))
Everything should work just fine after this change.
Note that you could also require the instance at the trait level by adding an (abstract) method like the following:
implicit def genA: Generic.Aux[A, T]
Then any class implementing LoadsCsv could have an implicit val genA parameter (or could supply the instance in some other way).
Scala 2.8.1
Take the following class hierarchy
abstract class A
class B extends A
class C extends A
Why is the scala compiler unable to find the implicit parameter for send when sending an instance of B below
implicit def routingKeyFor[T <: A](value: T) =
value.getClass.getSimpleName
implicit def routingKeyFor(value: C) = "custom C"
def send[T <: A](value: T)(implicit createRoutingKey: T => String):
Validation[Throwable, String] = Success(createRoutingKey(value))
val resultOfSendingB = send(new B)
val resultOfSendingC = send(new C)
Why is the compiler able to locate the value for the implicit parameter when the generic version of routingKeyFor is renamed?
implicit def someOtherName[T <: A](value: T) =
value.getClass.getSimpleName
The second implicit is shadowing the first one. Why is anyone's guess, and you might open an issue for it (after verifying that this wasn't reported before), but it might just be one of those things that throw a spanner into the works of type inference.