I have a constructor that has the following signature:
class Event[DL <: HList](
detailsIn: DL
)(implicit lbcr: LUBConstraint[DL, EventDetail[_]]) { ...
In the companion object I have:
def apply[DL <: HList](
detailIn: String*
)(implicit lbcr: LUBConstraint[DL, EventDetail[String]]) =
new Event(
if (detailIn.map(deet => EventDetail(deet)).toList.size == 1)
detailIn.map(deet => EventDetail(deet)).toList.toHList[String :: HNil].get
else throw new NotImplementedException()
)(lbcr)
Admittedly, this apply method could be cleaned up.
This yields the following error which I'm frankly lost about how to deal with:
Error:(87, 7) type mismatch;
found : shapeless.LUBConstraint[DL,edu.cornell.ansci.dairy.econ.model.event.EventDetail[String]]
required: shapeless.LUBConstraint[shapeless.::[String,shapeless.HNil],edu.cornell.ansci.dairy.econ.model.event.EventDetail[_]]
The second part of the issue: is there any way to make this polymorphic over the size of detailIn? Based on my other readings, I assume not, and I can think of no way to do it in Scala. The best I can do is to support for convenience a fixed number lengths for detailIn string sequences, and if that is exceeded, the user of the class must use HList directly.
Don't be scared by that type error, it's pretty explicit, let me try to type the body of your apply method bottom up:
At the very bottom you have a String :: HNil:
val e0: String :: HNil = e.toList.toHList[String :: HNil].get
The if doesn't change that:
val e1: String :: HNil =
if (???) e0
else throw new NotImplementedException()
Thus, when you create Event, Event.DL = String :: HNil:
new Event[String :: HNil](e1)(lbcr)
But lbcr has type LUBConstraint[DL, _], that's incompatible with LUBConstraint[String :: HNil, _]!
For the second part of your question, shapeless provides a safer alternative to varargs: ProductArgs. Backed by a macro and a slightly more involved definition, object myfunction extends ProductArgs { def applyProduct[L <: HList](args: L) } instead of def myfunction(args: Any*), you can get your varargs as an HList instead of a Seq[String].
Related
I am trying to learn shapeless, However I find that Shapeless code is really hard to understand.
So I got this code example from a talk given on youtube.
https://www.youtube.com/watch?v=JKaCCYZYBWo
Can anyone explain to me what is going on (step by step). The thing which I find most difficult is that everything is an implicit and so its really hard to trace the code ....
Also, please point me to some resources which can help me understand code like this. Everytime I encounter code with so many implicits I feel that I don't even know Scala.
import shapeless._
sealed trait Diff[A]
final case class Identical[A](value: A) extends Diff[A]
final case class Different[A](left : A, right : A) extends Diff[A]
object Diff {
def apply[A](left : A, right : A) : Diff[A] = {
if (left == right) Identical(left)
else Different(left, right)
}
}
trait SimpleDelta[R <: HList] extends DepFn2[R, R] {
type Out <: HList
}
object SimpleDelta {
type Aux[I <: HList, O <: HList] = SimpleDelta[I]{type Out = O}
implicit def hnilDelta: Aux[HNil, HNil] = new SimpleDelta[HNil] {
type Out = HNil
def apply(l : HNil, r: HNil) : Out = HNil
}
implicit def hconsDelta[H, T <: HList, DT <: HList](implicit tailDelta: Aux[T, DT])
: Aux[H::T, Diff[H] :: DT] = new SimpleDelta[H :: T] {
type Out = Diff[H] :: DT
def apply(l : H :: T, r: H :: T) : Out =
Diff(l.head, r.head) :: tailDelta(l.tail, r.tail)
}
def apply[A, R <: HList](l : A, r: A)
(implicit genA: Generic.Aux[A, R], delta: SimpleDelta[R]) : delta.Out =
delta(genA.to(l), genA.to(r))
}
case class Address(number: Int, street: String, city: String)
case class Character(name: String, age: Int, address: Address)
val homer = Character("Homer Simpson", 42, Address(742, "Evergreen Terrace", "SpringField"))
val ned = Character("Ned Flanders", 42, Address(744, "Evergreen Terrace", "SpringField"))
SimpleDelta(homer, ned)
The purpose of the program is to compare two instances of the same case class. This is done via heterogenous lists (HLists), which can function as generic representations of case classes. shapeless provides the infrastructure to convert between a case class and its generic HList representation. Note that in this example the difference is not calculated recursively, i.e. any nested case class instances will be compared atomically.
Let's assume the Diff trait and companion object are fairly self-explanatory.
The SimpleDelta trait defines a function with two arguments (DepFn2). The signature is (R, R) => Out. Furthermore, a constraint on the result type is declared: Out <: HList. This means that for all subtypes of SimpleDelta, the type member Out must be a subtype of HList.
The object SimpleDelta defines a type Aux. You find more info on the Aux pattern on the web, e.g. here. In short, it functions as an alias for the SimpleDelta trait, allowing to express the Out type member using a type parameter.
Now the SimpleDelta object defines two implicit methods hnilDelta and hconsDelta. Each of those methods represents one of the possible constuctors of HList, HNil and HCons. An HList A :: B :: HNil can also be read as ::(A, ::(B, HNil)) or HCons(A, HCons(B, HNil)). Using the two methods, the input HLists can be recursively deconstructed and processed. Based on the return types of these methods, the compiler knows which one to implicitly resolve when it encounters an implicit parameter of the type SimpleDelta[R]: When R has been inferred to HNil, it will invoke hnilDelta; when R has been inferred to Head :: Tail (or HCons(Head, Tail)) for some types Head and Tail, it will invoke hconsDelta.
The hnilDelta method determines the delta between two empty HLists (HNil is the empty HList).
Note that the Out type member is set to HNil, which is the type of the HNil object. The result is HNil, since there is no difference.
The return type of the method is Aux[HNil, HNil]. As explained above, this is an alias for SimpleDelta[HNil] { type Out = HNil }. It tells the compiler to invoke this method if it encounters an implicit parameter of type SimpleDelta[R], and R has been inferred to HNil.
The hconsDelta method determines the delta between two non-empty HLists., i.e. two HLists which are composed of a head element of type H and a tail list of type T.
The type parameter DT denotes the HList subtype representing the delta of the tail list.
Note the implicit parameter tailDelta :Aux[T, DT]. It conveys that the method must be able to determine the delta between the tails of the two lists l and r: tailDelta(l.tail, r.tail).
The return type of the method is Aux[H::T, Diff[H] :: DT]. As explained above, this is an alias for SimpleDelta[H::T] { type Out = Diff[H] :: DT }. It tells the compiler to invoke this method if it encounters an implicit parameter of type SimpleDelta[R], and R has been inferred to H::T for some types H and T.
The type member Out is set to Diff[H] :: DT, which is the HList type for the resulting difference.
The apply method determines the difference between the two head elements and prepends it to the difference betwee the tail lists, which is calculated using the implicit parameter tailDelta (see above).
The apply method takes care of converting the instances l and r of type A to their generic HList representations, and invokes a SimpleDelta on these HLists to calculate the difference.
The type R is the generic HList representation of A. The implicit parameter genA: Generic.Aux[A, R] denotes that the method requires a conversion between A and R.
Additionally, the method declares an implicit parameter delta: SimpleDelta[R]. This is required to calculate the difference between the two generic HList representations of l and r.
The return type of the method is delta.Out. Note that a dependent type is used here since the actual Out type member of the delta parameter is not known – it depends on the actual type A and its HList representation R.
What happens when SimpleDelta(homer, ned) is compiled?
The typed invocation is SimpleDelta.apply[Character, R](homer, ned).
The compiler tries to find matching values for the implicit parameters genA: Generic.Aux[Character, R] and delta: SimpleDelta[R].
shapeless provides the implicit Generic.Aux[A, R] for case classes A. This mechanism is rather complex and out of the scope of this question, but what is relevant is that the generic HList representation of type Character is String :: Int :: Address :: HNil, so the compiler can infer the type of genA to Generic.Aux[Character, String :: Int :: Address :: HNil]. This in turn means that the type variable R can be inferred to String :: Int :: Address :: HNil.
Now that the type R is known, the compiler will try to resolve the implicit value for parameter delta: SimpleDelta[R], i.e. SimpleDelta[String :: Int :: Address :: HNil]. The current scope doesn't contain any implicit methods with this return type or values with this type, but the compiler will also consult the implicit scope of the parameter type SimpleDelta, which includes the companion object SimpleDelta (see section Implicit Parameters in the Scala documentation).
The compiler searches the SimpleDelta object for values or methods which provide an object of type SimpleDelta[String :: Int :: Address :: HNil]. The only matching method is hconsDelta, so the compiler wires the parameter value delta to an invocation of this method. Note that the HList is decomposed into head and tail: hconsDelta[String, Int :: Address :: HNil, DT](implicit tailDelta: Aux[Int :: Address :: HNil, DT]).
To fill in the implicit tailDelta parameter, the compiler will continue to find a matching implicit method. This way, the HList is recursively deconstructed until only HNil is left and hnilDelta is invoked.
On the "way up", the DT type variable will be inferred to Diff[Int] :: Diff[Address] :: HNil, and the return type of hconsDelta to SimpleDelta[String :: Int :: Address :: HNil] {type Out = Diff[String] :: Diff[Int] :: Diff[Address] :: HNil]}. Thus, the return type of apply is inferred to Diff[String] :: Diff[Int] :: Diff[Address] :: HNil.
I am learning shapeless, and currently I am trying to create a function that does the following:
given a type of an HList it returns the HList of Nones, with the Option types corresponding to given HList type.
For instance:
create[String :: Int :: HNil] // returns None[String] :: None[Int] :: HNil
So the logic is the following:
def create[A <: HList] {
type HT = ??? //somehow getting Head type
type TT = ??? //somehow getting Tail type
// if HT is HNil HNil else Option.empty[HT] :: create[TT]
}
Looks like the HT and TT can be provided byIsHCons
def createHList[L <: HList](implicit ihc: IsHCons[L]): HList = {
type HT = ihc.H
type TT = ihc.T
//
}
But that rises two problems
How to compare types?
Compiler can not find IsHCons[TT] for recursive call. (How to get ISHCons[TT] from IsHCons[L]? It is not even possible for HNil!)
I think that I can get around the (1), by providing implicits for HNil and non HNil, so the compiler will pick up the right implicit, depending on the type.
Am I moving towards the right direction?
Given that, may be it is worth to ask more general question. Given the HList of Monoids, is it possible to derive zero HList, consisting of zeros of give monoids?
Thanks!
It's fairly easy to define Monoid instance for every HList where each element type has its Monoid instance:
trait Monoid[T] {
def zero: T
def plus(t1: T, t2: T): T
}
object Monoid {
implicit val HNilMonoid: Monoid[HNil] = new Monoid[HNil] {
def zero = HNil
def plus(hn1: HNil, hn2: HNil) = HNil
}
implicit def HConsMonoid[H, T <: HList](implicit hm: Monoid[H], tm: Monoid[T]): Monoid[H :: T] =
new Monoid[H :: T] {
def zero = hm.zero :: tm.zero
def plus(ht1: H :: T, ht2: H :: T) =
hm.plus(ht1.head, ht2.head) :: tm.plus(ht1.tail, ht2.tail)
}
}
(actually, I would expect shapeless to be able to derive the above automatically, but I'm not an expert on shapeless)
Now, assuming that we have Monoid[Int] and Monoid[String] defined elsewhere, you can just:
implicitly[Monoid[Int :: String :: HNil]].zero
which is exactly what you want, i.e. a HList of zeros.
I think I need a HList that is constrained to have all of its elements being a subtype of a certain type. LUBConstraint seems to be what I want, and indeed it does constrain the construction of such a HList - but I can't see how to get the evidence out again, so that I can map (actually, traverse, because it needs to be monadic) over the HList and call a method (that exists in the LUB type) on each of the elements.
In addition, I want the type of the HList resulting from the traverse operation to be exactly the same type as the type of the input HList.
The use case is a kind of functional "listener list" - all of the elements of the HList are "listeners" which must be notified of "events", accept or reject them, and return new versions of themselves with updated "internal state". If that was all I needed, then I could just use an ordinary immutable Scala collection. But I also want direct typed access to individual elements without using asInstanceOf - hence the motivation for trying to use HList.
In general if you have some operation that you want to perform on all of the elements in an HList, you'll want to map a polymorphic function value over the HList. For example, suppose I have the following setup:
trait Listener[L <: Listener[L]] {
def handle(s: String): Option[L]
}
class FooListener extends Listener[FooListener] {
def handle(s: String) =
if (s.size == 3) Some(this) else None
}
class BarListener extends Listener[BarListener ]{
def handle(s: String) = Some(this)
}
import shapeless._
val listeners = new FooListener :: new BarListener :: HNil
Now I want to send a String to each of these listeners and gather the results. If I just wanted to send a fixed value, this would be easy:
object event123 extends Poly1 {
implicit def listener[L <: Listener[L]] = at[L](_.handle("123"))
}
val result = listeners.map(event123)
Which will be appropriately typed as an Option[FooListener] :: Option[BarListener] :: HNil. If I'm using shapeless-contrib, I can sequence this HList:
import scalaz._, Scalaz._, shapeless.contrib.scalaz._
val sequenced: Option[FooListener :: BarListener :: HNil] = sequence(result)
Or just use traverse:
traverse(listeners)(event123)
Unfortunately there are restrictions on how polymorphic function values can be defined that mean that partial application isn't convenient, so if we don't know the String we're sending at compile time, this is a lot more complicated:
object event extends Poly1 {
implicit def listener[L <: Listener[L]] = at[(L, String)] {
case (listener, string) => listener.handle(string)
}
}
traverse(listeners.zip(listeners.mapConst("123")))(event)
Where we've zipped the elements with the string and then mapped a polymorphic function that takes tuples over the result. There are other ways you could do this using more or less the same approach, but none of them are terribly clear.
A completely different approach is just to skip the polymorphic function values and define a new type class:
trait Notifiable[L <: HList] {
def tell(s: String)(l: L): Option[L]
}
object Notifiable {
implicit val hnilNotifiable: Notifiable[HNil] = new Notifiable[HNil] {
def tell(s: String)(l: HNil) = Some(HNil)
}
implicit def hconsNotifiable[H <: Listener[H], T <: HList](implicit
tn: Notifiable[T]
): Notifiable[H :: T] = new Notifiable[H :: T] {
def tell(s: String)(l: H :: T) = for {
h <- l.head.handle(s)
t <- tn.tell(s)(l.tail)
} yield h :: t
}
}
def tell[L <: HList: Notifiable](s: String)(l: L) =
implicitly[Notifiable[L]].tell(s)(l)
And then:
val sequenced: Option[FooListener :: BarListener :: HNil] =
tell("123")(listeners)
This is less generic (it only works on Option, not arbitrary applicatives), but it doesn't require an extra dependency for sequencing, and it's arguably a little less muddled than jumping through hoops to partially apply a polymorphic function value because of weird limitations of the compiler.
Given the following definitions:
class R[T]
class A
class B
class C
This works:
val s1 = new R[A] :: new R[B] :: HNil
val r1 = s1.toList
// r1 of type: List[R[_ >: A with B]]
While this does not:
val s2 = new R[A] :: new R[B] :: new R[C] :: HNil
val r2 = s2.toList
// could not find implicit value for parameter toList:
// shapeless.ToList[shapeless.::[R[A],
// shapeless.::[R[B],shapeless.::[R[C],shapeless.HNil]]],Lub]
Where I expect:
// r2 of type: List[R[_ >: A with B with C]]
Pseudo solution:
Supply implicit yourself:
val r3 = s2.toList(ToList.hlistToList[R[A], R[B], ::[R[C], HNil],
R[_ >: A with B with C]])
This is of course not a solution, as it eliminates the whole benefit of HLists (the HList is supplied by a caller together with all necessary implicits).
Clarification
I am happy if I get a List[R[_]] at the end without the type bounds.
The code you've written ought to Just Work, as-is, producing the precise existential type that you were expecting. It'd be very helpful if you would open a bug against shapeless so that a fix for this doesn't get forgotten for the next release.
In the meantime, if the type you're really after is List[R[_]] (which seems more likely to be useful to me) then you can ask for it explicitly in a much simpler way than your workaround,
scala> import shapeless._
import shapeless._
scala> class R[T] ; class A ; class B ; class C
defined class R
defined class A
defined class B
defined class C
scala> val s2 = new R[A] :: new R[B] :: new R[C] :: HNil
s2: R[A] :: R[B] :: R[C] :: HNil = R#7a26bc5e :: R#518fdf9 :: R#2bc9e90c :: HNil
scala> s2.toList[R[_]]
res0: List[R[_]] = List(R#7a26bc5e, R#518fdf9, R#2bc9e90c)
Given that this less precise type is likely to be more useful in context, you'll want to continue using the explicit element type argument to toList even after the bug is fixed, so I consider this to be the right answer rather than a workaround.
I just submitted a PR that fixes this.
Just calling s2.toList, without explicitly specifying a type, should return the same type as if you created a (standard) list with the same items and let scalac infer the type of the list for you (that is something like List[R[_ >: A with B with C]]).
I have done a few implementations of HList now. One based on Daniel Spiewak's High Wizardry in the Land of Scala talk and another based on a post in Apocalisp blog. The goal was to have a heterogenous list of which is not heterogenous in the primary type but rather the higher kind. For example:
val requests = Request[String] :: Request[Int] :: HNil
I would be able to do a map across the list to perform the request and result in a heterogenous list of the higher kind. So:
requests.map(execute)
should equal
String :: Int :: HNil
Sadly all my attempts have resulted in an HList of Any. Here is the code from a recent attempt:
class Request[+Out](o:Out) {
type O = Out
def v:O = o
}
object HList {
trait Func[-Elem,Out] {
type Apply[E <: Elem] <: Out
def apply[N <: Elem](e:N):Apply[N]
}
sealed trait HList[Base] {
type Head <: Base
type Tail <: HList[Base]
type Map[Out,F <: Func[Base,Out]] <: HList[Out]
def head:Head
def tail:Tail
def ::[A <: Base](a:A):HList[Base]
def map[Out,F <: Func[Base,Out]](f:F):Map[Out,F]
}
case class HNil[Base]() extends HList[Base] {
type Head = Nothing
type Tail = Nothing
type Map[Out,F <: Func[Base,Out]] = HNil[Out]
def head = error("Head of an empty HList")
def tail = error("Head of an empty HList")
def ::[A <: Base](a:A) = HCons(a,this)
def map[Out,F <: Func[Base,Out]](f:F) = new HNil[Out]
}
case class HCons[Base,A <: Base,B <: HList[Base]](head: A, tail: B) extends HList[Base] {
type Head = A
type Tail = B
type Map[Out,F <: Func[Base,Out]] = HCons[Out,F#Apply[Head],Tail#Map[Out,F]]
def ::[C <: Base](c:C) = HCons(c,this)
def map[Out,F <: Func[Base,Out]](f:F) =
HCons(f(head),tail.map(f))
}
val :: = HCons
}
object Test extends Application {
import HList._
val HNil = new HNil[Request[_]]
val list = new Request[Int](1) :: new Request[String]("1") :: HNil
val (a :: b :: HNil) = list
val y:Request[String] = b
val results = list.map[Any,Unwrap.type](Unwrap)
val i:Int = results.head
}
import HList._
object Unwrap extends Func[Request[Any],Any] {
type Apply[I <: Request[Any]] = I#O
def apply[N <: Request[Any]](e:N) = null.asInstanceOf[Apply[N]]
}
The other attempt was based on the Apocalisp version which uses fold to create a new HList and again it resulted in a HList of Any types. Any tips would be appreciated.
The HList implementation in shapeless is rich enough to subsume both HList and KList functionality. It provides a map operation which applies a higher-ranked function, possibly with type-specific cases, across it's elements yielding an appropriately typed HList result,
import shapeless.Poly._
import shapeless.HList._
// Define a higher-ranked function from Sets to Options
object choose extends (Set ~> Option) {
def default[T](s : Set[T]) = s.headOption
}
// An HList of Sets
val sets = Set(1) :: Set("foo") :: HNil
// Map our choose function across it ...
val opts = sets map choose
// The resulting value
opts == Option(1) :: Option("foo") :: HNil
Note that although it's the case in the above example there's no requirement that the HList elements share a common outer type constructor, it just has to be the case that the higher-ranked function mapped with has cases for all of the types involved,
// size is a higher-ranked function from values of arbitrary type to a 'size'
// which is defined as 1 by default but which has type specific cases for
// Strings and tuples
object size extends (Id ~> Const[Int]#λ) {
def default[T](t : T) = 1
}
implicit def sizeString = size.λ[String](s => s.length)
implicit def sizeTuple[T, U](implicit st : size.λ[T], su : size.λ[U]) =
size.λ[(T, U)](t => 1+size(t._1)+size(t._2))
size(23) == 1 // Default
size("foo") == 3 // Type specific case for Strings
size((23, "foo")) == 5 // Type specific case for tuples
Now let's map this across an HList,
val l = 23 :: true :: "foo" :: ("bar", "wibble") :: HNil
val ls = l map size
ls == 1 :: 1 :: 3 :: 10 :: HNil
In this case the result type of the function being mapped is constant: it's an Int no matter what the argument type is. Consequently the resulting HList has elements all of the same type, which means that it can usefully be converted to a vanilla list,
ls.toList == List(1, 1, 3, 10)
what you need is a Klist with type constructor Request, and a natural transformation execute: Request ~> Id. All of this is detailed in the marvelous type-level programming series of posts at Apocalisp, in particular:
Natural transformation literals
Klist basics
you can checkout the code for the whole series from Mark Harrah's up repo
In your case, you'll need something like
val reqList = new Request[Int](1) :^: new Request[String]("1") :^: KNil
val exec = new (Request ~> Id) { def apply[T](reqs: Request[T]): T = reqs.execute }
val results = reqList down exec
the down method above is conceptually the same as map for a nat transf M ~> Id; you also have more general map which from a nat transf M ~> N and a Klist of kind M yields a KList of kind N.
Note that you have an example of Map with HList in the recent (October 2016, 5 years after the OP) article "Using shapeless' HLists for extra type safety (in Akka Streams)" from Mikołaj Koziarkiewicz.
//glue for the ParserStageDefs
specs.map(s => Flow[Data].map(s.parser).map(s.processor))
.foreach(broadcast ~> _ ~> merge)
The problem lies in the fact that the type information in our specs list is not preserved. Or rather, not preserved the way we want to - the type of the List elements is ParserStageDef[_ >: Int with String], so the lowest common supertype for our decorator and incrementer.
The above implies that, when mapping between the parser and processor elements, the compiler has no way to provide the actual type T that's used within the given spec.
A solution
Here's where HLists come to the rescue. Because they preserve the complete type information for each element, it's possible to define our flow very similarly to our last attempt.
First, let's replace our list with an HList:
import shapeless.ops.hlist._
import shapeless._
//...
val specs = decorator :: incrementer :: HNil
val specsSize = specs.length.toInt
Now, for the mapping from ParserStageDefs into Flows, we need to take a different approach, as the map for HList requires something called P**oly - a polymorphic function value**.
Here's how one would look like in our case:
import shapeless.PolyDefns.~>
object toFlow extends (ParserStageDef ~> ProcessingFlow) {
override def apply[T](f: ParserStageDef[T]) =
Flow[Data].map(f.parser).map(f.processor)
}
For it to work, we'll also have change ProcessingFlow to type ProcessingFlow[_] = Flow[Data, Data, _], since the polymorphic function above expects a higher-kinded type.
Now, our central statement turns out to be:
//we convert to a List[ProcessingFlow[_]] for simplicity
specs.map(toFlow).toList.foreach(broadcast ~> _ ~> merge)
and we're all set!