I am trying to learn shapeless, However I find that Shapeless code is really hard to understand.
So I got this code example from a talk given on youtube.
https://www.youtube.com/watch?v=JKaCCYZYBWo
Can anyone explain to me what is going on (step by step). The thing which I find most difficult is that everything is an implicit and so its really hard to trace the code ....
Also, please point me to some resources which can help me understand code like this. Everytime I encounter code with so many implicits I feel that I don't even know Scala.
import shapeless._
sealed trait Diff[A]
final case class Identical[A](value: A) extends Diff[A]
final case class Different[A](left : A, right : A) extends Diff[A]
object Diff {
def apply[A](left : A, right : A) : Diff[A] = {
if (left == right) Identical(left)
else Different(left, right)
}
}
trait SimpleDelta[R <: HList] extends DepFn2[R, R] {
type Out <: HList
}
object SimpleDelta {
type Aux[I <: HList, O <: HList] = SimpleDelta[I]{type Out = O}
implicit def hnilDelta: Aux[HNil, HNil] = new SimpleDelta[HNil] {
type Out = HNil
def apply(l : HNil, r: HNil) : Out = HNil
}
implicit def hconsDelta[H, T <: HList, DT <: HList](implicit tailDelta: Aux[T, DT])
: Aux[H::T, Diff[H] :: DT] = new SimpleDelta[H :: T] {
type Out = Diff[H] :: DT
def apply(l : H :: T, r: H :: T) : Out =
Diff(l.head, r.head) :: tailDelta(l.tail, r.tail)
}
def apply[A, R <: HList](l : A, r: A)
(implicit genA: Generic.Aux[A, R], delta: SimpleDelta[R]) : delta.Out =
delta(genA.to(l), genA.to(r))
}
case class Address(number: Int, street: String, city: String)
case class Character(name: String, age: Int, address: Address)
val homer = Character("Homer Simpson", 42, Address(742, "Evergreen Terrace", "SpringField"))
val ned = Character("Ned Flanders", 42, Address(744, "Evergreen Terrace", "SpringField"))
SimpleDelta(homer, ned)
The purpose of the program is to compare two instances of the same case class. This is done via heterogenous lists (HLists), which can function as generic representations of case classes. shapeless provides the infrastructure to convert between a case class and its generic HList representation. Note that in this example the difference is not calculated recursively, i.e. any nested case class instances will be compared atomically.
Let's assume the Diff trait and companion object are fairly self-explanatory.
The SimpleDelta trait defines a function with two arguments (DepFn2). The signature is (R, R) => Out. Furthermore, a constraint on the result type is declared: Out <: HList. This means that for all subtypes of SimpleDelta, the type member Out must be a subtype of HList.
The object SimpleDelta defines a type Aux. You find more info on the Aux pattern on the web, e.g. here. In short, it functions as an alias for the SimpleDelta trait, allowing to express the Out type member using a type parameter.
Now the SimpleDelta object defines two implicit methods hnilDelta and hconsDelta. Each of those methods represents one of the possible constuctors of HList, HNil and HCons. An HList A :: B :: HNil can also be read as ::(A, ::(B, HNil)) or HCons(A, HCons(B, HNil)). Using the two methods, the input HLists can be recursively deconstructed and processed. Based on the return types of these methods, the compiler knows which one to implicitly resolve when it encounters an implicit parameter of the type SimpleDelta[R]: When R has been inferred to HNil, it will invoke hnilDelta; when R has been inferred to Head :: Tail (or HCons(Head, Tail)) for some types Head and Tail, it will invoke hconsDelta.
The hnilDelta method determines the delta between two empty HLists (HNil is the empty HList).
Note that the Out type member is set to HNil, which is the type of the HNil object. The result is HNil, since there is no difference.
The return type of the method is Aux[HNil, HNil]. As explained above, this is an alias for SimpleDelta[HNil] { type Out = HNil }. It tells the compiler to invoke this method if it encounters an implicit parameter of type SimpleDelta[R], and R has been inferred to HNil.
The hconsDelta method determines the delta between two non-empty HLists., i.e. two HLists which are composed of a head element of type H and a tail list of type T.
The type parameter DT denotes the HList subtype representing the delta of the tail list.
Note the implicit parameter tailDelta :Aux[T, DT]. It conveys that the method must be able to determine the delta between the tails of the two lists l and r: tailDelta(l.tail, r.tail).
The return type of the method is Aux[H::T, Diff[H] :: DT]. As explained above, this is an alias for SimpleDelta[H::T] { type Out = Diff[H] :: DT }. It tells the compiler to invoke this method if it encounters an implicit parameter of type SimpleDelta[R], and R has been inferred to H::T for some types H and T.
The type member Out is set to Diff[H] :: DT, which is the HList type for the resulting difference.
The apply method determines the difference between the two head elements and prepends it to the difference betwee the tail lists, which is calculated using the implicit parameter tailDelta (see above).
The apply method takes care of converting the instances l and r of type A to their generic HList representations, and invokes a SimpleDelta on these HLists to calculate the difference.
The type R is the generic HList representation of A. The implicit parameter genA: Generic.Aux[A, R] denotes that the method requires a conversion between A and R.
Additionally, the method declares an implicit parameter delta: SimpleDelta[R]. This is required to calculate the difference between the two generic HList representations of l and r.
The return type of the method is delta.Out. Note that a dependent type is used here since the actual Out type member of the delta parameter is not known – it depends on the actual type A and its HList representation R.
What happens when SimpleDelta(homer, ned) is compiled?
The typed invocation is SimpleDelta.apply[Character, R](homer, ned).
The compiler tries to find matching values for the implicit parameters genA: Generic.Aux[Character, R] and delta: SimpleDelta[R].
shapeless provides the implicit Generic.Aux[A, R] for case classes A. This mechanism is rather complex and out of the scope of this question, but what is relevant is that the generic HList representation of type Character is String :: Int :: Address :: HNil, so the compiler can infer the type of genA to Generic.Aux[Character, String :: Int :: Address :: HNil]. This in turn means that the type variable R can be inferred to String :: Int :: Address :: HNil.
Now that the type R is known, the compiler will try to resolve the implicit value for parameter delta: SimpleDelta[R], i.e. SimpleDelta[String :: Int :: Address :: HNil]. The current scope doesn't contain any implicit methods with this return type or values with this type, but the compiler will also consult the implicit scope of the parameter type SimpleDelta, which includes the companion object SimpleDelta (see section Implicit Parameters in the Scala documentation).
The compiler searches the SimpleDelta object for values or methods which provide an object of type SimpleDelta[String :: Int :: Address :: HNil]. The only matching method is hconsDelta, so the compiler wires the parameter value delta to an invocation of this method. Note that the HList is decomposed into head and tail: hconsDelta[String, Int :: Address :: HNil, DT](implicit tailDelta: Aux[Int :: Address :: HNil, DT]).
To fill in the implicit tailDelta parameter, the compiler will continue to find a matching implicit method. This way, the HList is recursively deconstructed until only HNil is left and hnilDelta is invoked.
On the "way up", the DT type variable will be inferred to Diff[Int] :: Diff[Address] :: HNil, and the return type of hconsDelta to SimpleDelta[String :: Int :: Address :: HNil] {type Out = Diff[String] :: Diff[Int] :: Diff[Address] :: HNil]}. Thus, the return type of apply is inferred to Diff[String] :: Diff[Int] :: Diff[Address] :: HNil.
Related
I have a constructor that has the following signature:
class Event[DL <: HList](
detailsIn: DL
)(implicit lbcr: LUBConstraint[DL, EventDetail[_]]) { ...
In the companion object I have:
def apply[DL <: HList](
detailIn: String*
)(implicit lbcr: LUBConstraint[DL, EventDetail[String]]) =
new Event(
if (detailIn.map(deet => EventDetail(deet)).toList.size == 1)
detailIn.map(deet => EventDetail(deet)).toList.toHList[String :: HNil].get
else throw new NotImplementedException()
)(lbcr)
Admittedly, this apply method could be cleaned up.
This yields the following error which I'm frankly lost about how to deal with:
Error:(87, 7) type mismatch;
found : shapeless.LUBConstraint[DL,edu.cornell.ansci.dairy.econ.model.event.EventDetail[String]]
required: shapeless.LUBConstraint[shapeless.::[String,shapeless.HNil],edu.cornell.ansci.dairy.econ.model.event.EventDetail[_]]
The second part of the issue: is there any way to make this polymorphic over the size of detailIn? Based on my other readings, I assume not, and I can think of no way to do it in Scala. The best I can do is to support for convenience a fixed number lengths for detailIn string sequences, and if that is exceeded, the user of the class must use HList directly.
Don't be scared by that type error, it's pretty explicit, let me try to type the body of your apply method bottom up:
At the very bottom you have a String :: HNil:
val e0: String :: HNil = e.toList.toHList[String :: HNil].get
The if doesn't change that:
val e1: String :: HNil =
if (???) e0
else throw new NotImplementedException()
Thus, when you create Event, Event.DL = String :: HNil:
new Event[String :: HNil](e1)(lbcr)
But lbcr has type LUBConstraint[DL, _], that's incompatible with LUBConstraint[String :: HNil, _]!
For the second part of your question, shapeless provides a safer alternative to varargs: ProductArgs. Backed by a macro and a slightly more involved definition, object myfunction extends ProductArgs { def applyProduct[L <: HList](args: L) } instead of def myfunction(args: Any*), you can get your varargs as an HList instead of a Seq[String].
I am learning shapeless, and currently I am trying to create a function that does the following:
given a type of an HList it returns the HList of Nones, with the Option types corresponding to given HList type.
For instance:
create[String :: Int :: HNil] // returns None[String] :: None[Int] :: HNil
So the logic is the following:
def create[A <: HList] {
type HT = ??? //somehow getting Head type
type TT = ??? //somehow getting Tail type
// if HT is HNil HNil else Option.empty[HT] :: create[TT]
}
Looks like the HT and TT can be provided byIsHCons
def createHList[L <: HList](implicit ihc: IsHCons[L]): HList = {
type HT = ihc.H
type TT = ihc.T
//
}
But that rises two problems
How to compare types?
Compiler can not find IsHCons[TT] for recursive call. (How to get ISHCons[TT] from IsHCons[L]? It is not even possible for HNil!)
I think that I can get around the (1), by providing implicits for HNil and non HNil, so the compiler will pick up the right implicit, depending on the type.
Am I moving towards the right direction?
Given that, may be it is worth to ask more general question. Given the HList of Monoids, is it possible to derive zero HList, consisting of zeros of give monoids?
Thanks!
It's fairly easy to define Monoid instance for every HList where each element type has its Monoid instance:
trait Monoid[T] {
def zero: T
def plus(t1: T, t2: T): T
}
object Monoid {
implicit val HNilMonoid: Monoid[HNil] = new Monoid[HNil] {
def zero = HNil
def plus(hn1: HNil, hn2: HNil) = HNil
}
implicit def HConsMonoid[H, T <: HList](implicit hm: Monoid[H], tm: Monoid[T]): Monoid[H :: T] =
new Monoid[H :: T] {
def zero = hm.zero :: tm.zero
def plus(ht1: H :: T, ht2: H :: T) =
hm.plus(ht1.head, ht2.head) :: tm.plus(ht1.tail, ht2.tail)
}
}
(actually, I would expect shapeless to be able to derive the above automatically, but I'm not an expert on shapeless)
Now, assuming that we have Monoid[Int] and Monoid[String] defined elsewhere, you can just:
implicitly[Monoid[Int :: String :: HNil]].zero
which is exactly what you want, i.e. a HList of zeros.
I think I need a HList that is constrained to have all of its elements being a subtype of a certain type. LUBConstraint seems to be what I want, and indeed it does constrain the construction of such a HList - but I can't see how to get the evidence out again, so that I can map (actually, traverse, because it needs to be monadic) over the HList and call a method (that exists in the LUB type) on each of the elements.
In addition, I want the type of the HList resulting from the traverse operation to be exactly the same type as the type of the input HList.
The use case is a kind of functional "listener list" - all of the elements of the HList are "listeners" which must be notified of "events", accept or reject them, and return new versions of themselves with updated "internal state". If that was all I needed, then I could just use an ordinary immutable Scala collection. But I also want direct typed access to individual elements without using asInstanceOf - hence the motivation for trying to use HList.
In general if you have some operation that you want to perform on all of the elements in an HList, you'll want to map a polymorphic function value over the HList. For example, suppose I have the following setup:
trait Listener[L <: Listener[L]] {
def handle(s: String): Option[L]
}
class FooListener extends Listener[FooListener] {
def handle(s: String) =
if (s.size == 3) Some(this) else None
}
class BarListener extends Listener[BarListener ]{
def handle(s: String) = Some(this)
}
import shapeless._
val listeners = new FooListener :: new BarListener :: HNil
Now I want to send a String to each of these listeners and gather the results. If I just wanted to send a fixed value, this would be easy:
object event123 extends Poly1 {
implicit def listener[L <: Listener[L]] = at[L](_.handle("123"))
}
val result = listeners.map(event123)
Which will be appropriately typed as an Option[FooListener] :: Option[BarListener] :: HNil. If I'm using shapeless-contrib, I can sequence this HList:
import scalaz._, Scalaz._, shapeless.contrib.scalaz._
val sequenced: Option[FooListener :: BarListener :: HNil] = sequence(result)
Or just use traverse:
traverse(listeners)(event123)
Unfortunately there are restrictions on how polymorphic function values can be defined that mean that partial application isn't convenient, so if we don't know the String we're sending at compile time, this is a lot more complicated:
object event extends Poly1 {
implicit def listener[L <: Listener[L]] = at[(L, String)] {
case (listener, string) => listener.handle(string)
}
}
traverse(listeners.zip(listeners.mapConst("123")))(event)
Where we've zipped the elements with the string and then mapped a polymorphic function that takes tuples over the result. There are other ways you could do this using more or less the same approach, but none of them are terribly clear.
A completely different approach is just to skip the polymorphic function values and define a new type class:
trait Notifiable[L <: HList] {
def tell(s: String)(l: L): Option[L]
}
object Notifiable {
implicit val hnilNotifiable: Notifiable[HNil] = new Notifiable[HNil] {
def tell(s: String)(l: HNil) = Some(HNil)
}
implicit def hconsNotifiable[H <: Listener[H], T <: HList](implicit
tn: Notifiable[T]
): Notifiable[H :: T] = new Notifiable[H :: T] {
def tell(s: String)(l: H :: T) = for {
h <- l.head.handle(s)
t <- tn.tell(s)(l.tail)
} yield h :: t
}
}
def tell[L <: HList: Notifiable](s: String)(l: L) =
implicitly[Notifiable[L]].tell(s)(l)
And then:
val sequenced: Option[FooListener :: BarListener :: HNil] =
tell("123")(listeners)
This is less generic (it only works on Option, not arbitrary applicatives), but it doesn't require an extra dependency for sequencing, and it's arguably a little less muddled than jumping through hoops to partially apply a polymorphic function value because of weird limitations of the compiler.
Suppose I have an abitrary KList, which for the sake of argument has type constructor Option[_], ie;
type Example = Option[Int] :: Option[String] :: HNil
Is there a way I can retrieve an Hlist made of the type parameters?
type Params = Int :: String :: HNil
So, for example I might be able to define some sort of arbitrary getOrElse method
getOrElse(ex:Example, default:Params):Params
Now I'm looking for something possibly of a form like this (or similar as the type structure I propose might not be feasible).
case class MyOptionList[L <: HList](maybes:L) {
type Concrete = {somehow the hlist params types as an Hlist}
def getOrElse(default:Concrete):Concrete = ???
}
I'm not Miles, but it's possible to accomplish what you're trying to do pretty elegantly with Shapeless's Comapped:
import shapeless._, ops.hlist.Comapped
case class MyOptionList[L <: HList, C <: HList](maybes: L)(
implicit val comapped: Comapped.Aux[L, Option, C]
) {
def getOrElse(default: C): C = default // Not a useful implementation
}
And then:
scala> val x: Int :: HNil = MyOptionList(Option(1) :: HNil).getOrElse(2 :: HNil)
x: shapeless.::[Int,shapeless.HNil] = 2 :: HNil
Note that in some cases it can be more convenient to put the constraint on the method:
case class MyOptionList[L <: HList](maybes: L) {
def getOrElse[C <: HList: ({ type l[x] = Comapped.Aux[L, Option, x] })#l](
default: C
): C = default
}
Here the usage is the same, but you don't have the extra type parameter on the case class. If you want to use this approach but constrain the creation of MyOptionList to disallow non-Option members, you could use L <: HList: *->*[Option]#λ in its type parameter list.
#MilesSabin will probably show up with a more elegant answer, but you can construct this recursively by hand, the way many of the Shapeless internal things are written:
sealed trait GetOrElse[L <: HList] {
type Concrete <: HList
def getOrElse(ex: L, default: Concrete): Concrete
}
object GetOrElse {
implicit def nil = new GetOrElse[HNil]{
type Concrete = HNil
def getOrElse(ex: HNil, default: HNil) = HNil
}
implicit def cons[H, T <: HList](implicit tl: GetOrElse[T]) =
new GetOrElse[Option[H] :: T]{
type Concrete = H :: tl.Concrete
def getOrElse(ex: Option[H] :: T, default: Concrete) =
ex.head.getOrElse(default.head) ::
tl.getOrElse(ex.tail, default.tail)
}
def apply[L <: HList](implicit goe: GetOrElse[L])
: GetOrElse[L]{type Concrete = goe.Concrete} = goe
}
case class MyOptionList[L <: HList, C](maybes:L)(
implicit goe: GetOrElse[L]{type Concrete = C}) {
def getOrElse(default:C):C = goe.getOrElse(maybes, default)
}
(It's probably possible to use a type member rather than a type parameter, but I get confused about when the type information for those is eaten, so I tend to prefer to use type parameters everywhere)
Given the following definitions:
class R[T]
class A
class B
class C
This works:
val s1 = new R[A] :: new R[B] :: HNil
val r1 = s1.toList
// r1 of type: List[R[_ >: A with B]]
While this does not:
val s2 = new R[A] :: new R[B] :: new R[C] :: HNil
val r2 = s2.toList
// could not find implicit value for parameter toList:
// shapeless.ToList[shapeless.::[R[A],
// shapeless.::[R[B],shapeless.::[R[C],shapeless.HNil]]],Lub]
Where I expect:
// r2 of type: List[R[_ >: A with B with C]]
Pseudo solution:
Supply implicit yourself:
val r3 = s2.toList(ToList.hlistToList[R[A], R[B], ::[R[C], HNil],
R[_ >: A with B with C]])
This is of course not a solution, as it eliminates the whole benefit of HLists (the HList is supplied by a caller together with all necessary implicits).
Clarification
I am happy if I get a List[R[_]] at the end without the type bounds.
The code you've written ought to Just Work, as-is, producing the precise existential type that you were expecting. It'd be very helpful if you would open a bug against shapeless so that a fix for this doesn't get forgotten for the next release.
In the meantime, if the type you're really after is List[R[_]] (which seems more likely to be useful to me) then you can ask for it explicitly in a much simpler way than your workaround,
scala> import shapeless._
import shapeless._
scala> class R[T] ; class A ; class B ; class C
defined class R
defined class A
defined class B
defined class C
scala> val s2 = new R[A] :: new R[B] :: new R[C] :: HNil
s2: R[A] :: R[B] :: R[C] :: HNil = R#7a26bc5e :: R#518fdf9 :: R#2bc9e90c :: HNil
scala> s2.toList[R[_]]
res0: List[R[_]] = List(R#7a26bc5e, R#518fdf9, R#2bc9e90c)
Given that this less precise type is likely to be more useful in context, you'll want to continue using the explicit element type argument to toList even after the bug is fixed, so I consider this to be the right answer rather than a workaround.
I just submitted a PR that fixes this.
Just calling s2.toList, without explicitly specifying a type, should return the same type as if you created a (standard) list with the same items and let scalac infer the type of the list for you (that is something like List[R[_ >: A with B with C]]).