I want to have a generic list of different types and I want to get objects with specific type.
I have this structure:
trait Parent
case class A() extends Parent
case class B() extends Parent
case class C() extends Parent
I may define different lists with different size of:
val list:List[Parent] = A() :: B() :: C() :: C() ::Nil //or any size or ordering
I want to cast this in runtime to Hlist of A,B,C,C. .toHlist[] need types of elements which should be infer in runtime.
What would be Shapeless way of it? Is there any way to convert it to tuples?
You can't do it. Note that
should be infer in runtime
is a contradiction. Type inference works in compile time only.
In other words, when you write something like
val list: A :: B :: C :: C :: HNil = ...
the type of list variable is known at compile time. It is impossible to assign a type to a variable at runtime, it just doesn't make sense. Suppose it would be possible, and you would have a magical method toHlistMagical:
val list: List[Parent] = A() :: B() :: C() :: C() :: Nil
val hlist = list.toHlistMagical // infers to A :: B :: C :: C :: HNil
Now let's change it a bit:
def getHlist(list: List[Parent]) = list.toHlistMagical
Now what return type do you expect for this function to have? Bear in mind that it can be called with various lists, not only those which contain A, B, C, C instances in this order:
getHlist(C() :: B() :: A() :: A() :: Nil)
It could be Any, but then you can just go with List anyway because there is no HList anymore, so you don't get any extra type safety.
That's the reason why toHlist needs types:
def getHlist(list: List[Parent]) = list.toHlist[A :: B :: C :: C :: HNil]
getHlist now has return type Option[A :: B :: C :: C :: HNil], because you specified the exact type you want, and toHlist will be able to perform runtime validation of the list structure and return Some if the list does actually contain these types in this order or None if it does not.
The following function can convert your list to a HList:
def listToHList(x: List[_]): HList = {
if(x == Nil) HNil
else x.head :: listToHList(x.tail)
}
For your sample, you can use it like the following:
listToHList(list).asInstanceOf[A :: B :: C :: HNil]
Related
I am trying to learn shapeless, However I find that Shapeless code is really hard to understand.
So I got this code example from a talk given on youtube.
https://www.youtube.com/watch?v=JKaCCYZYBWo
Can anyone explain to me what is going on (step by step). The thing which I find most difficult is that everything is an implicit and so its really hard to trace the code ....
Also, please point me to some resources which can help me understand code like this. Everytime I encounter code with so many implicits I feel that I don't even know Scala.
import shapeless._
sealed trait Diff[A]
final case class Identical[A](value: A) extends Diff[A]
final case class Different[A](left : A, right : A) extends Diff[A]
object Diff {
def apply[A](left : A, right : A) : Diff[A] = {
if (left == right) Identical(left)
else Different(left, right)
}
}
trait SimpleDelta[R <: HList] extends DepFn2[R, R] {
type Out <: HList
}
object SimpleDelta {
type Aux[I <: HList, O <: HList] = SimpleDelta[I]{type Out = O}
implicit def hnilDelta: Aux[HNil, HNil] = new SimpleDelta[HNil] {
type Out = HNil
def apply(l : HNil, r: HNil) : Out = HNil
}
implicit def hconsDelta[H, T <: HList, DT <: HList](implicit tailDelta: Aux[T, DT])
: Aux[H::T, Diff[H] :: DT] = new SimpleDelta[H :: T] {
type Out = Diff[H] :: DT
def apply(l : H :: T, r: H :: T) : Out =
Diff(l.head, r.head) :: tailDelta(l.tail, r.tail)
}
def apply[A, R <: HList](l : A, r: A)
(implicit genA: Generic.Aux[A, R], delta: SimpleDelta[R]) : delta.Out =
delta(genA.to(l), genA.to(r))
}
case class Address(number: Int, street: String, city: String)
case class Character(name: String, age: Int, address: Address)
val homer = Character("Homer Simpson", 42, Address(742, "Evergreen Terrace", "SpringField"))
val ned = Character("Ned Flanders", 42, Address(744, "Evergreen Terrace", "SpringField"))
SimpleDelta(homer, ned)
The purpose of the program is to compare two instances of the same case class. This is done via heterogenous lists (HLists), which can function as generic representations of case classes. shapeless provides the infrastructure to convert between a case class and its generic HList representation. Note that in this example the difference is not calculated recursively, i.e. any nested case class instances will be compared atomically.
Let's assume the Diff trait and companion object are fairly self-explanatory.
The SimpleDelta trait defines a function with two arguments (DepFn2). The signature is (R, R) => Out. Furthermore, a constraint on the result type is declared: Out <: HList. This means that for all subtypes of SimpleDelta, the type member Out must be a subtype of HList.
The object SimpleDelta defines a type Aux. You find more info on the Aux pattern on the web, e.g. here. In short, it functions as an alias for the SimpleDelta trait, allowing to express the Out type member using a type parameter.
Now the SimpleDelta object defines two implicit methods hnilDelta and hconsDelta. Each of those methods represents one of the possible constuctors of HList, HNil and HCons. An HList A :: B :: HNil can also be read as ::(A, ::(B, HNil)) or HCons(A, HCons(B, HNil)). Using the two methods, the input HLists can be recursively deconstructed and processed. Based on the return types of these methods, the compiler knows which one to implicitly resolve when it encounters an implicit parameter of the type SimpleDelta[R]: When R has been inferred to HNil, it will invoke hnilDelta; when R has been inferred to Head :: Tail (or HCons(Head, Tail)) for some types Head and Tail, it will invoke hconsDelta.
The hnilDelta method determines the delta between two empty HLists (HNil is the empty HList).
Note that the Out type member is set to HNil, which is the type of the HNil object. The result is HNil, since there is no difference.
The return type of the method is Aux[HNil, HNil]. As explained above, this is an alias for SimpleDelta[HNil] { type Out = HNil }. It tells the compiler to invoke this method if it encounters an implicit parameter of type SimpleDelta[R], and R has been inferred to HNil.
The hconsDelta method determines the delta between two non-empty HLists., i.e. two HLists which are composed of a head element of type H and a tail list of type T.
The type parameter DT denotes the HList subtype representing the delta of the tail list.
Note the implicit parameter tailDelta :Aux[T, DT]. It conveys that the method must be able to determine the delta between the tails of the two lists l and r: tailDelta(l.tail, r.tail).
The return type of the method is Aux[H::T, Diff[H] :: DT]. As explained above, this is an alias for SimpleDelta[H::T] { type Out = Diff[H] :: DT }. It tells the compiler to invoke this method if it encounters an implicit parameter of type SimpleDelta[R], and R has been inferred to H::T for some types H and T.
The type member Out is set to Diff[H] :: DT, which is the HList type for the resulting difference.
The apply method determines the difference between the two head elements and prepends it to the difference betwee the tail lists, which is calculated using the implicit parameter tailDelta (see above).
The apply method takes care of converting the instances l and r of type A to their generic HList representations, and invokes a SimpleDelta on these HLists to calculate the difference.
The type R is the generic HList representation of A. The implicit parameter genA: Generic.Aux[A, R] denotes that the method requires a conversion between A and R.
Additionally, the method declares an implicit parameter delta: SimpleDelta[R]. This is required to calculate the difference between the two generic HList representations of l and r.
The return type of the method is delta.Out. Note that a dependent type is used here since the actual Out type member of the delta parameter is not known – it depends on the actual type A and its HList representation R.
What happens when SimpleDelta(homer, ned) is compiled?
The typed invocation is SimpleDelta.apply[Character, R](homer, ned).
The compiler tries to find matching values for the implicit parameters genA: Generic.Aux[Character, R] and delta: SimpleDelta[R].
shapeless provides the implicit Generic.Aux[A, R] for case classes A. This mechanism is rather complex and out of the scope of this question, but what is relevant is that the generic HList representation of type Character is String :: Int :: Address :: HNil, so the compiler can infer the type of genA to Generic.Aux[Character, String :: Int :: Address :: HNil]. This in turn means that the type variable R can be inferred to String :: Int :: Address :: HNil.
Now that the type R is known, the compiler will try to resolve the implicit value for parameter delta: SimpleDelta[R], i.e. SimpleDelta[String :: Int :: Address :: HNil]. The current scope doesn't contain any implicit methods with this return type or values with this type, but the compiler will also consult the implicit scope of the parameter type SimpleDelta, which includes the companion object SimpleDelta (see section Implicit Parameters in the Scala documentation).
The compiler searches the SimpleDelta object for values or methods which provide an object of type SimpleDelta[String :: Int :: Address :: HNil]. The only matching method is hconsDelta, so the compiler wires the parameter value delta to an invocation of this method. Note that the HList is decomposed into head and tail: hconsDelta[String, Int :: Address :: HNil, DT](implicit tailDelta: Aux[Int :: Address :: HNil, DT]).
To fill in the implicit tailDelta parameter, the compiler will continue to find a matching implicit method. This way, the HList is recursively deconstructed until only HNil is left and hnilDelta is invoked.
On the "way up", the DT type variable will be inferred to Diff[Int] :: Diff[Address] :: HNil, and the return type of hconsDelta to SimpleDelta[String :: Int :: Address :: HNil] {type Out = Diff[String] :: Diff[Int] :: Diff[Address] :: HNil]}. Thus, the return type of apply is inferred to Diff[String] :: Diff[Int] :: Diff[Address] :: HNil.
I am learning shapeless, and currently I am trying to create a function that does the following:
given a type of an HList it returns the HList of Nones, with the Option types corresponding to given HList type.
For instance:
create[String :: Int :: HNil] // returns None[String] :: None[Int] :: HNil
So the logic is the following:
def create[A <: HList] {
type HT = ??? //somehow getting Head type
type TT = ??? //somehow getting Tail type
// if HT is HNil HNil else Option.empty[HT] :: create[TT]
}
Looks like the HT and TT can be provided byIsHCons
def createHList[L <: HList](implicit ihc: IsHCons[L]): HList = {
type HT = ihc.H
type TT = ihc.T
//
}
But that rises two problems
How to compare types?
Compiler can not find IsHCons[TT] for recursive call. (How to get ISHCons[TT] from IsHCons[L]? It is not even possible for HNil!)
I think that I can get around the (1), by providing implicits for HNil and non HNil, so the compiler will pick up the right implicit, depending on the type.
Am I moving towards the right direction?
Given that, may be it is worth to ask more general question. Given the HList of Monoids, is it possible to derive zero HList, consisting of zeros of give monoids?
Thanks!
It's fairly easy to define Monoid instance for every HList where each element type has its Monoid instance:
trait Monoid[T] {
def zero: T
def plus(t1: T, t2: T): T
}
object Monoid {
implicit val HNilMonoid: Monoid[HNil] = new Monoid[HNil] {
def zero = HNil
def plus(hn1: HNil, hn2: HNil) = HNil
}
implicit def HConsMonoid[H, T <: HList](implicit hm: Monoid[H], tm: Monoid[T]): Monoid[H :: T] =
new Monoid[H :: T] {
def zero = hm.zero :: tm.zero
def plus(ht1: H :: T, ht2: H :: T) =
hm.plus(ht1.head, ht2.head) :: tm.plus(ht1.tail, ht2.tail)
}
}
(actually, I would expect shapeless to be able to derive the above automatically, but I'm not an expert on shapeless)
Now, assuming that we have Monoid[Int] and Monoid[String] defined elsewhere, you can just:
implicitly[Monoid[Int :: String :: HNil]].zero
which is exactly what you want, i.e. a HList of zeros.
What I am trying to achieve is a way to:
Define the list of types for a heterogeneous list
From the definition above, build a statically typed list of values
Ideally I would like to type the following expression in the IDE:
val record = types.addValue("test").addValue(123).addValue(new java.util.Date())
having the type of the arguments to addValue() constrained by the IDE type inference engine.
What follows is a working implementation of the point 1: a specification of record fields types:
case class FieldType[V, T <: FieldType[_, _]](clazz: Class[V], tail: T) {
def addValue(value: V) = FieldValue(value, tail)
}
case class FieldValue[V, T <: FieldType[_, _]](value: V, tail: T)
object emptyFieldType extends FieldType(classOf[Null], null)
and this is an example specification of a record made of String, Int, Date:
val types = FieldType(
classOf[String], FieldType(
classOf[Int], FieldType(
classOf[java.util.Date], emptyFieldType
)
)
)
By using the addValue method in the FieldType type, the compiler recognises the argument types at any depth:
val a = types.addValue("") // here only String allowed
val b = types.tail.addValue(23) // here only Int allowed
val c = types.tail.tail.addValue(new java.util.Date()) // here only Date allowed
But....
I haven't yet found a way to implement a forwarding method on the FieldValue type, to achieve a fluent interface as in my first example:
val record = types.addValue("test").addValue(123).addValue(new java.util.Date())
This is a pseudo-code to give the idea:
case class FieldValue[V, T <: FieldType[_, _]](value: V, tail: T) {
def addValue(x: What Here??) = tail.addValue(x) // not compiling!
}
I think it must be possible, since the type information for the next argument to addValue is contained in FieldValue via the tail member, as V. But I could not find a way to make this information available to the addValue method, to let the compiler validate the type of the argument value, and the IDE to suggest the correct type.
You may want to take a look at shapeless.
Here's a trivial example demonstrating how you can statically specify the type of an heterogeneous list:
type R = String :: Int :: java.util.Date :: HNil
val record: R = "test" :: 123 :: new java.util.Date() :: HNil
// record: R = test :: 123 :: Thu Aug 14 00:21:52 CEST 2014 :: HNil
val record: R = "test" :: "foo" :: new java.util.Date() :: HNil
// error: type mismatch;
found : shapeless.::[String,shapeless.::[String,shapeless.::[java.util.Date,shapeless.HNil]]]
required: R
(which expands to) shapeless.::[String,shapeless.::[Int,shapeless.::[java.util.Date,shapeless.HNil]]]
val record: R = "test" :: "hola" :: new java.util.Date() :: HNil
^
I don't know whether this already fits your needs, but in any case you should definitely check all the features of shapeless, as it provides many facilities for this kind of generic programming
Given the following definitions:
class R[T]
class A
class B
class C
This works:
val s1 = new R[A] :: new R[B] :: HNil
val r1 = s1.toList
// r1 of type: List[R[_ >: A with B]]
While this does not:
val s2 = new R[A] :: new R[B] :: new R[C] :: HNil
val r2 = s2.toList
// could not find implicit value for parameter toList:
// shapeless.ToList[shapeless.::[R[A],
// shapeless.::[R[B],shapeless.::[R[C],shapeless.HNil]]],Lub]
Where I expect:
// r2 of type: List[R[_ >: A with B with C]]
Pseudo solution:
Supply implicit yourself:
val r3 = s2.toList(ToList.hlistToList[R[A], R[B], ::[R[C], HNil],
R[_ >: A with B with C]])
This is of course not a solution, as it eliminates the whole benefit of HLists (the HList is supplied by a caller together with all necessary implicits).
Clarification
I am happy if I get a List[R[_]] at the end without the type bounds.
The code you've written ought to Just Work, as-is, producing the precise existential type that you were expecting. It'd be very helpful if you would open a bug against shapeless so that a fix for this doesn't get forgotten for the next release.
In the meantime, if the type you're really after is List[R[_]] (which seems more likely to be useful to me) then you can ask for it explicitly in a much simpler way than your workaround,
scala> import shapeless._
import shapeless._
scala> class R[T] ; class A ; class B ; class C
defined class R
defined class A
defined class B
defined class C
scala> val s2 = new R[A] :: new R[B] :: new R[C] :: HNil
s2: R[A] :: R[B] :: R[C] :: HNil = R#7a26bc5e :: R#518fdf9 :: R#2bc9e90c :: HNil
scala> s2.toList[R[_]]
res0: List[R[_]] = List(R#7a26bc5e, R#518fdf9, R#2bc9e90c)
Given that this less precise type is likely to be more useful in context, you'll want to continue using the explicit element type argument to toList even after the bug is fixed, so I consider this to be the right answer rather than a workaround.
I just submitted a PR that fixes this.
Just calling s2.toList, without explicitly specifying a type, should return the same type as if you created a (standard) list with the same items and let scalac infer the type of the list for you (that is something like List[R[_ >: A with B with C]]).
I have done a few implementations of HList now. One based on Daniel Spiewak's High Wizardry in the Land of Scala talk and another based on a post in Apocalisp blog. The goal was to have a heterogenous list of which is not heterogenous in the primary type but rather the higher kind. For example:
val requests = Request[String] :: Request[Int] :: HNil
I would be able to do a map across the list to perform the request and result in a heterogenous list of the higher kind. So:
requests.map(execute)
should equal
String :: Int :: HNil
Sadly all my attempts have resulted in an HList of Any. Here is the code from a recent attempt:
class Request[+Out](o:Out) {
type O = Out
def v:O = o
}
object HList {
trait Func[-Elem,Out] {
type Apply[E <: Elem] <: Out
def apply[N <: Elem](e:N):Apply[N]
}
sealed trait HList[Base] {
type Head <: Base
type Tail <: HList[Base]
type Map[Out,F <: Func[Base,Out]] <: HList[Out]
def head:Head
def tail:Tail
def ::[A <: Base](a:A):HList[Base]
def map[Out,F <: Func[Base,Out]](f:F):Map[Out,F]
}
case class HNil[Base]() extends HList[Base] {
type Head = Nothing
type Tail = Nothing
type Map[Out,F <: Func[Base,Out]] = HNil[Out]
def head = error("Head of an empty HList")
def tail = error("Head of an empty HList")
def ::[A <: Base](a:A) = HCons(a,this)
def map[Out,F <: Func[Base,Out]](f:F) = new HNil[Out]
}
case class HCons[Base,A <: Base,B <: HList[Base]](head: A, tail: B) extends HList[Base] {
type Head = A
type Tail = B
type Map[Out,F <: Func[Base,Out]] = HCons[Out,F#Apply[Head],Tail#Map[Out,F]]
def ::[C <: Base](c:C) = HCons(c,this)
def map[Out,F <: Func[Base,Out]](f:F) =
HCons(f(head),tail.map(f))
}
val :: = HCons
}
object Test extends Application {
import HList._
val HNil = new HNil[Request[_]]
val list = new Request[Int](1) :: new Request[String]("1") :: HNil
val (a :: b :: HNil) = list
val y:Request[String] = b
val results = list.map[Any,Unwrap.type](Unwrap)
val i:Int = results.head
}
import HList._
object Unwrap extends Func[Request[Any],Any] {
type Apply[I <: Request[Any]] = I#O
def apply[N <: Request[Any]](e:N) = null.asInstanceOf[Apply[N]]
}
The other attempt was based on the Apocalisp version which uses fold to create a new HList and again it resulted in a HList of Any types. Any tips would be appreciated.
The HList implementation in shapeless is rich enough to subsume both HList and KList functionality. It provides a map operation which applies a higher-ranked function, possibly with type-specific cases, across it's elements yielding an appropriately typed HList result,
import shapeless.Poly._
import shapeless.HList._
// Define a higher-ranked function from Sets to Options
object choose extends (Set ~> Option) {
def default[T](s : Set[T]) = s.headOption
}
// An HList of Sets
val sets = Set(1) :: Set("foo") :: HNil
// Map our choose function across it ...
val opts = sets map choose
// The resulting value
opts == Option(1) :: Option("foo") :: HNil
Note that although it's the case in the above example there's no requirement that the HList elements share a common outer type constructor, it just has to be the case that the higher-ranked function mapped with has cases for all of the types involved,
// size is a higher-ranked function from values of arbitrary type to a 'size'
// which is defined as 1 by default but which has type specific cases for
// Strings and tuples
object size extends (Id ~> Const[Int]#λ) {
def default[T](t : T) = 1
}
implicit def sizeString = size.λ[String](s => s.length)
implicit def sizeTuple[T, U](implicit st : size.λ[T], su : size.λ[U]) =
size.λ[(T, U)](t => 1+size(t._1)+size(t._2))
size(23) == 1 // Default
size("foo") == 3 // Type specific case for Strings
size((23, "foo")) == 5 // Type specific case for tuples
Now let's map this across an HList,
val l = 23 :: true :: "foo" :: ("bar", "wibble") :: HNil
val ls = l map size
ls == 1 :: 1 :: 3 :: 10 :: HNil
In this case the result type of the function being mapped is constant: it's an Int no matter what the argument type is. Consequently the resulting HList has elements all of the same type, which means that it can usefully be converted to a vanilla list,
ls.toList == List(1, 1, 3, 10)
what you need is a Klist with type constructor Request, and a natural transformation execute: Request ~> Id. All of this is detailed in the marvelous type-level programming series of posts at Apocalisp, in particular:
Natural transformation literals
Klist basics
you can checkout the code for the whole series from Mark Harrah's up repo
In your case, you'll need something like
val reqList = new Request[Int](1) :^: new Request[String]("1") :^: KNil
val exec = new (Request ~> Id) { def apply[T](reqs: Request[T]): T = reqs.execute }
val results = reqList down exec
the down method above is conceptually the same as map for a nat transf M ~> Id; you also have more general map which from a nat transf M ~> N and a Klist of kind M yields a KList of kind N.
Note that you have an example of Map with HList in the recent (October 2016, 5 years after the OP) article "Using shapeless' HLists for extra type safety (in Akka Streams)" from Mikołaj Koziarkiewicz.
//glue for the ParserStageDefs
specs.map(s => Flow[Data].map(s.parser).map(s.processor))
.foreach(broadcast ~> _ ~> merge)
The problem lies in the fact that the type information in our specs list is not preserved. Or rather, not preserved the way we want to - the type of the List elements is ParserStageDef[_ >: Int with String], so the lowest common supertype for our decorator and incrementer.
The above implies that, when mapping between the parser and processor elements, the compiler has no way to provide the actual type T that's used within the given spec.
A solution
Here's where HLists come to the rescue. Because they preserve the complete type information for each element, it's possible to define our flow very similarly to our last attempt.
First, let's replace our list with an HList:
import shapeless.ops.hlist._
import shapeless._
//...
val specs = decorator :: incrementer :: HNil
val specsSize = specs.length.toInt
Now, for the mapping from ParserStageDefs into Flows, we need to take a different approach, as the map for HList requires something called P**oly - a polymorphic function value**.
Here's how one would look like in our case:
import shapeless.PolyDefns.~>
object toFlow extends (ParserStageDef ~> ProcessingFlow) {
override def apply[T](f: ParserStageDef[T]) =
Flow[Data].map(f.parser).map(f.processor)
}
For it to work, we'll also have change ProcessingFlow to type ProcessingFlow[_] = Flow[Data, Data, _], since the polymorphic function above expects a higher-kinded type.
Now, our central statement turns out to be:
//we convert to a List[ProcessingFlow[_]] for simplicity
specs.map(toFlow).toList.foreach(broadcast ~> _ ~> merge)
and we're all set!