Can someone explain me what the difference between these two approaches for typeclass instance derivation (specifically for Option[A])?
1.
trait MyTrait[A] {...}
object MyTrait extends LowPriority {
// instances for primitives
}
trait LowPriority extends LowestPriority {
final implicit def generic[A, H <: HList](
implicit gen: Generic.Aux[A, H],
h: Lazy[MyTrait[H]]
): MyTrait[A] = ???
final implicit val hnil: MyTrait[HNil] = ???
final implicit def product[H, T <: HList](
implicit h: Lazy[MyTrait[H]],
t: Lazy[MyTrait[T]]
): MyTrait[H :: T] = ???
}
// special instances for Options
trait LowestPriority {
implicit def genericOption[A, Repr <: HList](
implicit gen: Generic.Aux[A, Repr],
hEncoder: Lazy[MyTrait[Option[Repr]]]
): MyTrait[Option[A]] = ???
implicit val hnilOption: MyTrait[Option[HNil]] = ???
implicit def productOption1[H, T <: HList](
implicit
head: Lazy[MyTrait[Option[H]]],
tail: Lazy[MyTrait[Option[T]]],
notOption: H <:!< Option[Z] forSome { type Z }
): MyTrait[Option[H :: T]] = ???
implicit def product2[H, T <: HList](
implicit
head: Lazy[MyTrait[Option[H]]],
tail: Lazy[MyTrait[Option[T]]
): MyTrait[Option[Option[H] :: T]] = ???
}
trait MyTrait[A] {...}
object MyTrait extends LowPriority {
// instances for primitives
}
trait LowPriority {
// deriving instances for options from existing non-option instances
final implicit def forOption[A](implicit instance: MyTrait[A]): MyTrait[Option[A]] = ??? // <<<----
final implicit def generic[A, H <: HList](
implicit gen: Generic.Aux[A, H],
h: Lazy[MyTrait[H]]
): MyTrait[A] = ???
final implicit val hnil: MyTrait[HNil] = ???
final implicit def product[H, T <: HList](
implicit h: Lazy[MyTrait[H]],
t: Lazy[MyTrait[T]]
): MyTrait[H :: T] = ???
}
I tried both and they worked correctly, but i'm not sure that they will produce the same results for all cases (maybe i've missed something).
Do we really need LowestPriority instances for this?
Am i right if i would say that the first approach gives us just a little bit more flexibility?
I assuming that by "worked correctly" you mean "compiled" or "worked for some simple use case".
Both of your examples deal with generic product types, but not with generic sum types, so there is no risk that e.g. Option[A] could get derived using Some[A] :+: None :+: CNil, which would enforce some ambiguity. So (as far as I can tell) you could write the second version like:
trait MyTrait[A] {...}
object MyTrait extends LowPriority {
// instances for primitives
// deriving instances for options from existing non-option instances
final implicit def forOption[A](implicit instance: MyTrait[A]): MyTrait[Option[A]] = ???
}
trait LowPriority {
// <<<----
final implicit def hcons[A, H <: HList](
implicit gen: Generic.Aux[A, H],
h: Lazy[MyTrait[H]]
): MyTrait[A] = ???
final implicit val hnil: MyTrait[HNil] = ???
final implicit def product[H, T <: HList](
implicit h: Lazy[MyTrait[H]],
t: Lazy[MyTrait[T]]
): MyTrait[H :: T] = ???
}
and it would derive things correctly.
But how 1. and 2. differs?
In second version you can derive MyTrait[Option[A]] if you can derive for A, and you can derive for any A which is primitive/option/product - so Option[Option[String]], Option[String] and Option[SomeCaseClass] should all work. It should also work if this SomeCaseClass contains fields which are Options, or other case classes which are Options, etc.
Version 1. is slightly different:
at first you are looking for primitives
then you try to derive for a product (so e.g. Option would not be handled here)
then you do something weird:
genericOption assumes that you created a Option[Repr], and then I guess map it using Repr
in order to build that Repr you take Option[HNil] and prepend types inside Option using productOption, which would break if someone used Option as a field
so you "fix" that by prepending an Option in a special case product2
I guess, you tested that only against case classes, because the first version would not work for:
Option for primitives (Option[String], Option[Int] or whatever you defined as primitive)
nested options (Option[Option[String]])
options for custom defined types which are not case classes but have manually defined instances:
class MyCustomType
object MyCustomType {
implicit val myTrait: MyTrait[MyCustomType]
}
implicitly[Option[MyCustomType]]
For that reason any solution with implicit def forOption[A](implicit instance: MyTrait[A]): MyTrait[Option[A]] is simpler and more bulletproof.
Depending on what you put directly into companion low-priority implicits might be or might not be needed:
if you defined coproducts then manual support for e.g. Option, List, Either could conflict with shapeless derived ones
if you manually implemented MyTrait implicit for some type in its companion object then it would have the same priority as implicits directly in MyTrait - so if it could be derived using shapeless you could have conflicts
For that reason it makes sense to put shapeless implicits in LowPriorityImplicits but primitives, and manual codecs for List, Option, Either, etc directly in companion. That is, unless you defined some e.g. Option[String] implicits directly in companion which could clash with "Option[A] with implicit for A".
Since I don't know your exact use case I cannot tell for sure, but I would probably go with the seconds approach, or most likely with the one I implemented in the snippet above.
Actually it's hard to say without right hand sides and actual implementations.
From information you provided it doesn't follow that the two type classes behave equivalently.
For example in the 1st approach you consider some special cases, so theoretically it's possible that you redefine some general behavior in special case differently.
By the way, Option[A] is a coproduct of Some[A] and None.type (List[A] is a coproduct of scala.::[A] and Nil.type) and sometimes it's easier to derive a type class for coproducts than for Option[A] (or List[A]).
Related
I want to derive instances of type classes from unary case classes. But when i try to implicitly derive it i always get an error message. If i derive it explicitly using implicit method - it works. I'm not sure, but maybe the reason is that i missed some implicit types in my function
import shapeless._
import scala.reflect.ClassTag
import scala.reflect.runtime.universe.TypeTag
sealed trait Foo[A] {
def hello(): Unit
}
object Foo {
def apply[A](implicit foo: Foo[A]): foo.type = foo
def instance[A](implicit tag: ClassTag[A]): Foo[A] = new Foo[A] {
override def hello(): Unit = println(s"Hello from ${tag.runtimeClass.getName}")
}
}
trait Instances extends LowestPriority {
implicit val intHelloInstance: Foo[Int] = Foo.instance[Int]
}
trait LowestPriority {
implicit def derive[A: TypeTag, L <: HList, H](
implicit gen: Generic.Aux[A, L],
H: Lazy[Foo[H]],
isUnary: (H :: HNil) =:= L
): Foo[A] =
new Foo[A] {
override def hello(): Unit = {
print(s"Derived: ")
H.value.hello()
}
}
}
object T extends Instances {
case class A(a: Int)
def main(args: Array[String]): Unit = {
intHelloInstance.hello()
// val a: Foo[A] = derive[A, Int :: HNil, Int] // this works
// a.hello()
Foo[A].hello() // error
}
}
From logs:
Information:(45, 8) shapeless.this.Generic.materialize is not a valid
implicit value for shapeless.Generic.Aux[H,L] because:
hasMatchingSymbol reported error: H is not a case class, case
class-like, a sealed trait or Unit
Foo[A].hello()
How can i fix it?
This is one of the cases when behavior depends on... the order of implicits to resolve.
If you modify signature to:
implicit def derive[A, L <: HList, H](
implicit gen: Generic.Aux[A, L],
isUnary: (H :: HNil) =:= L, // swapped
H: Lazy[Foo[H]] // with this
): Foo[A] = ...
compiler will:
try to find some HList L that could be paired with A
then prove that L equal to some H :: HNil figuring out H in the process
finally using that H to fetch Lazy[Foo[H]]
and you will successfully compile Foo[A].hello().
When these two last argument are swapped to what you have in your question, compiler has to
assume some H (which in our test case most likely WON"T be Int)
then forcefully try to adjust L to match it
then go back to Generic which now is forced to prove that A is representable by some H :: HNil where H is most likely not Int and failing to do it but with a misguiding error information
This is one of these cases that shows that shapeless based derivation is sometimes tricky, and also here it shows that shapeless authors assumed in macros that the most likely cause of macro expansion failure is that A was not case class, while as we just saw it might be also compiler forcing some impossible proof because type inference got things wrong because we resolved them in wrong order.
If I have a method such as:
def f[T: Generic, U: Generic](t: T): U
Generic[T].to(t) returns type Generic[T]#Repr which I assume is a type alias for some type of HList.
Is it possible to select members from the HList and build another HList which I can convince the compiler is of type Generic[U]#Repr which I can then use to create an instance of U using Generic[U].from(myNewHList)?
I have tried many approaches but seem to be going around in circles.
When doing stuff like this in Shapeless, the best place to look is in the shapeless.ops typeclasses. In this case, since you know your second class is a strict subset of your first, Intersection is sufficient to get what you want. You'll want to set this up as a type class, so that you can pass in your input and output types and let the compiler infer the intermediate stuff.
trait Converter[A,B] {
def convert(a: A): B
}
object Converter {
def apply[A,B](implicit converter: Converter[A,B]) = converter
implicit def genericConverter[A, B, ARepr <: HList, BRepr <: HList](
implicit
genA: Generic.Aux[A,ARepr],
genB: Generic.Aux[B,BRepr],
intersection: shapeless.ops.hlist.Intersection.Aux[ARepr,BRepr,BRepr]
): Converter[A,B] =
new Converter[A,B]{def convert(a: A): B = genB.from(intersection(genA.to(a)))}
}
This can be used as follows:
case class Foo(a: Int, b: Int, c: String)
case class Bar(a: Int, c: String)
val foo = Foo(1,2,"Three")
val bar: Bar = Converter[Foo, Bar].convert(foo)
In the following code I try to derive typeclass instances with shapeless. However, in the case of a more complex case class (which is converted to a more complex HList), the compiler gives me a "diverging implicit expansion" even though it does not seem to resolve the same kind of implicit type twice. Maybe I am missing some other rule of the compiler?
(Fiddle: https://scalafiddle.io/sf/WEpnAXN/0)
import shapeless._
trait TC[T]
sealed trait Trait1
case class SimpleClass(a: String) extends Trait1
sealed trait Trait2
case class ComplexClass(a: String, b: String) extends Trait2
object Serialization extends App {
//Instances for HList
implicit val hnilInstance: TC[HNil] = ???
implicit def hconsInstance[H, T <: HList] (implicit t: TC[T]): TC[H :: T] = ???
//Instances for CoProduct
implicit val cnilInstance: TC[CNil] = ???
implicit def cconsInstance[H, T <: Coproduct] (implicit h: TC[H], t: TC[T]): TC[H :+: T] = ???
//Instances for Generic, relying on HNil & HCons
implicit def genericInstance[T, H] (implicit g: Generic.Aux[T, H], t: TC[H]): TC[T] = ???
the[TC[SimpleClass :+: CNil]] //Works
the[TC[Trait1]] //Works
the[TC[ComplexClass :+: CNil]] //Works
the[TC[Trait2]] //Fails with diverging implicit expansion
}
When trying to resolve the[TC[Trait1]] the compiler should do something like that:
TC[Trait1]
Generic[Trait1]
TC[SimpleClass :+: CNil]
TC[SimpleClass]
Generic[SimpleClass]
TC[String :: HNil]
TC[CNil]
which seems to work. However, with the 2-field case class, the compiler fails to do something like this - so I wonder: why do I have to use Lazy here already to make it work?
TC[Trait2]
Generic[Trait2]
TC[ComplexClass :+: CNil]
TC[ComplexClass]
Generic[ComplexClass]
TC[String :: String :: HNil]
TC[CNil]
I have created some fiddle so you can execute the code there directy.
A couple of years ago when I was working through some issues like this I found that the easiest way to figure out what the divergence checker was doing was just to throw some printlns into the compiler and publish it locally. In 2.12 the relevant code is the dominates method here, where we can replace the last line with something like this:
overlaps(dtor1, dted1) && (dtor1 =:= dted1 || {
val dtorC = complexity(dtor1)
val dtedC = complexity(dted1)
val result = dtorC > dtedC
println(if (result) "Dominates:" else "Does not dominate:")
println(s"$dtor (complexity: $dtorC)")
println(s"$dted (complexity: $dtedC)")
println("===========================")
result
})
Then we can sbt publishLocal scalac and try to compile your code:
Dominates:
TC[shapeless.::[String,shapeless.::[String,shapeless.HNil]]] (complexity: 7)
TC[shapeless.:+:[ComplexClass,shapeless.CNil]] (complexity: 6)
===========================
The problem here is that we're looking for a TC instance for String :: String :: HNil (the lowest node in your tree), but we have an open search for ComplexClass :+: CNil (three steps up). The compiler thinks that String :: String :: HNil both overlaps and dominates ComplexClass :+: CNil, and it bails out because that looks recursive.
This sounds ridiculous, so we can do an experiment to try to convince ourselves by adding some complexity to the coproduct part and seeing what happens. Let's just add a constructor:
case class Foo(i: Int) extends Trait2
Now everything works fine, and we get this message during compilation:
Does not dominate:
TC[shapeless.::[String,shapeless.::[String,shapeless.HNil]]] (complexity: 7)
TC[shapeless.:+:[ComplexClass,shapeless.:+:[Foo,shapeless.CNil]]] (complexity: 9)
So the ComplexClass hlist representation still overlaps the Trait2 coproduct representation, but it doesn't dominate it, since the Trait2 representation (the subject of the open implicit TC search we're worried about) is now more complex.
The checker is clearly being much too paranoid here, and its behavior may change in the future, but for now we're stuck with it. As you note, the most straightforward and fool-proof workaround is to stick a Lazy in there to hide the supposed recursion from the divergence checker.
In this case specifically, though, it looks like just putting the instances in the TC companion object works as well:
import shapeless._
sealed trait Trait1
case class SimpleClass(a: String) extends Trait1
sealed trait Trait2
case class ComplexClass(a: String, b: String) extends Trait2
trait TC[T]
object TC {
//Instances for HList
implicit def hnilInstance: TC[HNil] = ???
implicit def hconsInstance[H, T <: HList](implicit t: TC[T]): TC[H :: T] = ???
//Instances for CoProduct
implicit def cnilInstance: TC[CNil] = ???
implicit def cconsInstance[H, T <: Coproduct](implicit
h: TC[H], t: TC[T]
): TC[H :+: T] = ???
//Instances for Generic, relying on HNil & HCons
implicit def genericInstance[T, H](implicit
g: Generic.Aux[T, H], t: TC[H]
): TC[T] = ???
}
And then:
Does not dominate:
TC[shapeless.::[String,shapeless.::[String,shapeless.HNil]]] (complexity: 7)
TC[shapeless.:+:[ComplexClass,shapeless.CNil]] (complexity: 16)
Why does moving stuff around like this raise the complexity of the coproduct? I have no idea.
I've got the following code to basically iterate through the fields of a case class and map them using a Poly to the same type and use ToList[HL, Out]
For simplicity we can assume the Poly does this:
object Schema extends Poly1 {
implicit def caseInt = at[Int](_ => "I'm an int")
implicit def caseString = at[String](_ => "Oh boy a string")
}
def infer[V1 <: Product, Out <: HList, MapperOut <: HList](v1: V1)(
implicit gen: Generic.Aux[V1, Out],
map: Mapper.Aux[Schema.type, Out, MapperOut],
to: ToList[MapperOut, String]
): List[String] = to (gen to v1 map Schema)
This is all very straightforward and works very well for simple scenarios:
case class Test(id: Int, text: String)
val list = infer(Test(2, "text"))
// List("I'm an int", "Oh boy a string")
Now going out to where the buses don't run:
class Automagical[T <: Product with Serializable : TypeTag] {
def instance: T
// The typetag bit is needed for something else
def convert: List[String] = infer(instance)
}
Sadly any call to above fails with:
could not find implicit value for parameter gen: shapeless.Generic.Aux[T,Out]
Bonus
How can I improve the infer method by not requiring an instance of T at all? Obviously type inference is fine, but I do need to somehow materialise a List[String] from an HList[Lub] and map over something.
Given I only ever care about the types, is it possible to derive a concrete instance of List[String] by only knowing the types to be poly mapped encoded as an HList?
Something like:
def infer[V1 <: Product, Out <: HList, MapperOut <: HList]()(
implicit gen: Generic.Aux[V1, Out],
map: Mapper.Aux[Schema.type, Out, MapperOut],
to: ToList[MapperOut, String]
): List[String] = {
// Magically build an HList with samples from its types.
// Or an alternative approach that gives the same outcome
val reifiedInstance = reify[Out]
to (reifiedInstance map Schema)
}
To ensure that convert has access to the implicit parameters for infer, they have to be present when the Automagical instance is created:
abstract class Automagical[T <: Product with Serializable : TypeTag,
Out <: HList, MapperOut <: HList]
(implicit
gen: Generic.Aux[T, Out],
map: Mapper.Aux[Schema.type, Out, MapperOut],
to: ToList[MapperOut, String]) {
def instance: T
// The typetag bit is needed for something else
def convert: List[String] = infer(instance)
}
I want to find out if a particular class member is present in a given case class or not.
Following does give me that answer, failing at compile time which is right. (credit to Travis Brown)
scala> def getIntId[A, R <: HList](a: A)(implicit
| gen: LabelledGeneric.Aux[A, R],
| sel: Selector.Aux[R, Witness.`'id`.T, Int]
| ): Int = sel(gen.to(a))
case class Foo(id: String)
case class Bar(id: Int, name: String)
scala> getIntId(Bar(123, "bar"))
res3: Int = 123
scala> getIntId(Foo("12345"))
<console>:15: error: could not find implicit value for parameter sel: shapeless.ops.record.Selector.Aux[R,Symbol with shapeless.tag.Tagged[String("id")],Int]
getIntId(Foo("12345"))
Now if I don't have a concrete type but T to pass into getIntId method, is there a way to get this to work with generic type T ?
Update
Say have a method called findIdFromType as described below which takes in type T (where T will always be a some case class). is it possible to make this work?
def findIdFromType[T](t:T) = {
getIntId(t) //as expected doesn't compile
}
getIntId requires two implicit parameters.
If you need to call it in the context of a local method, you need to prove that those parameters exist in such context.
In order to do so, you have to propagate the implicit evidence, like this
def findIdFromType[A, R <: HList](a: A)(implicit
gen: LabelledGeneric.Aux[A, R],
sel: Selector.Aux[R, Witness.`'id`.T, Int]): Int = getIntId(a)
This of course is a completely useless example, but in case you want to perform other operations inside the wrapping method, this is the way to go: propagate the implicit evidence.