Evidence-preserving LUB constraint for HList - scala

I think I need a HList that is constrained to have all of its elements being a subtype of a certain type. LUBConstraint seems to be what I want, and indeed it does constrain the construction of such a HList - but I can't see how to get the evidence out again, so that I can map (actually, traverse, because it needs to be monadic) over the HList and call a method (that exists in the LUB type) on each of the elements.
In addition, I want the type of the HList resulting from the traverse operation to be exactly the same type as the type of the input HList.
The use case is a kind of functional "listener list" - all of the elements of the HList are "listeners" which must be notified of "events", accept or reject them, and return new versions of themselves with updated "internal state". If that was all I needed, then I could just use an ordinary immutable Scala collection. But I also want direct typed access to individual elements without using asInstanceOf - hence the motivation for trying to use HList.

In general if you have some operation that you want to perform on all of the elements in an HList, you'll want to map a polymorphic function value over the HList. For example, suppose I have the following setup:
trait Listener[L <: Listener[L]] {
def handle(s: String): Option[L]
}
class FooListener extends Listener[FooListener] {
def handle(s: String) =
if (s.size == 3) Some(this) else None
}
class BarListener extends Listener[BarListener ]{
def handle(s: String) = Some(this)
}
import shapeless._
val listeners = new FooListener :: new BarListener :: HNil
Now I want to send a String to each of these listeners and gather the results. If I just wanted to send a fixed value, this would be easy:
object event123 extends Poly1 {
implicit def listener[L <: Listener[L]] = at[L](_.handle("123"))
}
val result = listeners.map(event123)
Which will be appropriately typed as an Option[FooListener] :: Option[BarListener] :: HNil. If I'm using shapeless-contrib, I can sequence this HList:
import scalaz._, Scalaz._, shapeless.contrib.scalaz._
val sequenced: Option[FooListener :: BarListener :: HNil] = sequence(result)
Or just use traverse:
traverse(listeners)(event123)
Unfortunately there are restrictions on how polymorphic function values can be defined that mean that partial application isn't convenient, so if we don't know the String we're sending at compile time, this is a lot more complicated:
object event extends Poly1 {
implicit def listener[L <: Listener[L]] = at[(L, String)] {
case (listener, string) => listener.handle(string)
}
}
traverse(listeners.zip(listeners.mapConst("123")))(event)
Where we've zipped the elements with the string and then mapped a polymorphic function that takes tuples over the result. There are other ways you could do this using more or less the same approach, but none of them are terribly clear.
A completely different approach is just to skip the polymorphic function values and define a new type class:
trait Notifiable[L <: HList] {
def tell(s: String)(l: L): Option[L]
}
object Notifiable {
implicit val hnilNotifiable: Notifiable[HNil] = new Notifiable[HNil] {
def tell(s: String)(l: HNil) = Some(HNil)
}
implicit def hconsNotifiable[H <: Listener[H], T <: HList](implicit
tn: Notifiable[T]
): Notifiable[H :: T] = new Notifiable[H :: T] {
def tell(s: String)(l: H :: T) = for {
h <- l.head.handle(s)
t <- tn.tell(s)(l.tail)
} yield h :: t
}
}
def tell[L <: HList: Notifiable](s: String)(l: L) =
implicitly[Notifiable[L]].tell(s)(l)
And then:
val sequenced: Option[FooListener :: BarListener :: HNil] =
tell("123")(listeners)
This is less generic (it only works on Option, not arbitrary applicatives), but it doesn't require an extra dependency for sequencing, and it's arguably a little less muddled than jumping through hoops to partially apply a polymorphic function value because of weird limitations of the compiler.

Related

How to create a constrained HList from a sequence?

I have a constructor that has the following signature:
class Event[DL <: HList](
detailsIn: DL
)(implicit lbcr: LUBConstraint[DL, EventDetail[_]]) { ...
In the companion object I have:
def apply[DL <: HList](
detailIn: String*
)(implicit lbcr: LUBConstraint[DL, EventDetail[String]]) =
new Event(
if (detailIn.map(deet => EventDetail(deet)).toList.size == 1)
detailIn.map(deet => EventDetail(deet)).toList.toHList[String :: HNil].get
else throw new NotImplementedException()
)(lbcr)
Admittedly, this apply method could be cleaned up.
This yields the following error which I'm frankly lost about how to deal with:
Error:(87, 7) type mismatch;
found : shapeless.LUBConstraint[DL,edu.cornell.ansci.dairy.econ.model.event.EventDetail[String]]
required: shapeless.LUBConstraint[shapeless.::[String,shapeless.HNil],edu.cornell.ansci.dairy.econ.model.event.EventDetail[_]]
The second part of the issue: is there any way to make this polymorphic over the size of detailIn? Based on my other readings, I assume not, and I can think of no way to do it in Scala. The best I can do is to support for convenience a fixed number lengths for detailIn string sequences, and if that is exceeded, the user of the class must use HList directly.
Don't be scared by that type error, it's pretty explicit, let me try to type the body of your apply method bottom up:
At the very bottom you have a String :: HNil:
val e0: String :: HNil = e.toList.toHList[String :: HNil].get
The if doesn't change that:
val e1: String :: HNil =
if (???) e0
else throw new NotImplementedException()
Thus, when you create Event, Event.DL = String :: HNil:
new Event[String :: HNil](e1)(lbcr)
But lbcr has type LUBConstraint[DL, _], that's incompatible with LUBConstraint[String :: HNil, _]!
For the second part of your question, shapeless provides a safer alternative to varargs: ProductArgs. Backed by a macro and a slightly more involved definition, object myfunction extends ProductArgs { def applyProduct[L <: HList](args: L) } instead of def myfunction(args: Any*), you can get your varargs as an HList instead of a Seq[String].

Using Shapeless to implement an (HList of Function, Input) => Tuple typeclass

I've got some existing code along the lines of
trait Field[T]
object Fields {
case object Id extends Field[Int]
case object Name extends Field[String]
// ... and so on
}
// basically just a Map[Field[_], Any]
class QueryResultData {
def apply[T](field: Field[T]): T
}
def query(fields: Set[Field]): QueryMonad[QueryResultData]
So for example if I want to query the Id and Name data, I need to do something like:
val idsAndNames = for {
results <- query(Set(Fields.Id, Fields.Name))
} yield {
val id = results(Fields.Id)
val name = results(Fields.Name)
(id, name)
}
Having to manually extract each field's result is tedious, especially when the query includes more fields. What I'd like to be able to do is:
val idsAndNames: QueryMonad[(Int, String)] = query(Fields.Id -> Fields.Name)
And have some kind of typeclass handle the val id = ... part and reconstruct the tuple for me, e.g.
def query[Fields <: HList, Tuple](fields: Fields)
(implicit extractor: Extractor[Fields, T])
: QueryMonad[T]
How can I implement the Extractor typeclass so that I don't have to manually extract results?
What I've Tried
I figured this was a job for Shapeless, as the query method is meant to work on any number of fields, and is expected to give me back an appropriate tuple.
I defined a FieldExtractor type:
class FieldExtractor[T](field: Field[T]) {
def apply(results: QueryResultData): T = results(field)
}
and a polymorphic function for Field to FieldExtractor:
object makeFieldExtractor extends (Field ~> FieldExtractor) {
def apply[T](field: Field[T]) = new FieldExtractor[T]
}
and for simplicity's sake I'll start by dealing with HLists instead of Tuples:
val someFields = Fields.Id :: Fields.Name :: Fields.OtherStuff :: HNil
I tried using my makeFieldExtractor to convert someFields into someFieldExtractors. This is where I started running into trouble.
val someFieldExtractors = someFields.map(makeFieldExtractor)
error: could not find implicit value for parameter mapper: shapeless.ops.hlist.Mapper[MakeFieldExtractor.type,shapeless.::[Fields.Id.type,shapeless.::[Fields.Name.type,shapeless.::[Fields.OtherStuff.type,shapeless.HNil]]]]
It seems like the problem is that it's seeing types like Fields.Id.type when it probably should be seeing Field[Int]. If I explicitly specify the field types for someFields, the map works, but I don't want client code to have to do that. The compiler should do that for me. And let's assume I can't just change the Id/Name definitions to a val instead of a case object.
I found https://github.com/milessabin/shapeless/blob/master/examples/src/main/scala/shapeless/examples/klist.scala but didn't manage to make any successful use of it.
Here is how I would done it.
import shapeless.{::, HList, HNil}
import Field._
trait Field[A]
object Field {
case object IntField extends Field[Int]
case object StringField extends Field[String]
// Here is a little trick to proof that for any T that
// happened to be a subclass of Field[A] the Out is A
implicit def fieldExtractor[T, A]
(implicit ev: T <:< Field[A]): Extractor.Aux[T, A] =
new Extractor[T] {
override type Out = A
}
}
// The extractor for A
trait Extractor[A] {
type Out // Produces result of type Out
}
object Extractor {
// The Aux pattern http://gigiigig.github.io/posts/2015/09/13/aux-pattern.html
type Aux[A, Out0] = Extractor[A] {
type Out = Out0
}
// Proof that Out for HNil is HNil
implicit val hnilExtractor: Aux[HNil, HNil] =
new Extractor[HNil] {
override type Out = HNil
}
// Proof that Out for T :: H is hlist of extractor result for H and T
implicit def hconsExtractor[H, HO, T <: HList, TO <: HList]
(implicit H: Aux[H, HO], T: Aux[T, TO]): Aux[H :: T, HO :: TO] =
new Extractor[H :: T] {
override type Out = HO :: TO
}
}
type QueryMonad[A] = A
// Use dependent type Out as a result
def query[Fields](fields: Fields)(implicit extractor: Extractor[Fields]): QueryMonad[extractor.Out] = ???
val result = query(IntField :: StringField :: HNil)

Polymorphic mapping function using shapeless

I am trying to make this work in a type-safe manner:
val rows = db.select( ID_COLUMN, STR("name"), INT("count") ). from("tablename") ......
for ( (id, name, count) <- rows ) {
//some code to use the individual values
}
I have not found, so far, another way of making this type-safe besides the shapeless.
I do know about slick and other ORMs, so please don't send me there.
HList seems to be the way to approach passing in a heterogeneous bunch of objects and get back a list of values with specific types.
I tried to do something like this :
trait ColDef [X] {
def colName
def valalue (m:Map[String, Object]):X
}
object XS extends ColDef[String]{
def colName = "s"
def value (m:Map[String, Object]) = m("s").asInstanceOf[String]
}
object XI extends ColDef[Integer]{
def colName = "i"
def value (m:Map[String, Object]) = m("i").asInstanceOf[Integer]
}
val xhl = XS::XI::HNil
val data:Map[String,Object] = Map(("s" ->"asdf"), ("i" -> new Integer(5)))
object p1 extends Poly1 {
implicit def getValue[T, S <% ColDef[T]] = at[S] (coldef => coldef.value(data) )
}
val res = xhl map p1
val (s, i) = res.tupled //this works, giving s:String, and i:Integer
//but following does not compile
def nextstep(hl : HList, data:Map[String,Object]) = {
hl map p1
}
Just to reiterate what is essential:
HList/Shapeless are likely candidates for solving the problem, but are not the goal of this exercise. My goal is to have the return type of a function correspond to the variable types and number of parameters passed in.
It would be ideal if the user of my little utility did not need to know about HList, but that is not a real requirement.
The essential part is having the type of the result match the type of params :
val param1 = new Conf[String]
val param2 = new Conf[Integer]
... etc ....
val res = function(param1, param2, param3)
More precisely payload types of the parameters above, so that the type of res is T(String, Integer, ....) .
Let me add another clarification. I want to create a method for arbitrary arity and avoid creating a function for each count of parameters. If I was ok with 22 methods, it would look something like this:
def f[A](a:ColDef[A]):(A)
def f[A,B](a:ColDef[A], b:ColDef[B]):(A,B)
def f[A,B,C](a:ColDef[A], b:ColDef[B],c:ColDef[C]):(A,B,C)
..... and so on
And then I would not need shapeless or HList as all the possible tuples would be explicitly defined.
Actually looking at those 3 signatures - making them 22 would take a bit of time, but would avoid shapeless dependency an their implementations would also be one-liners. May be I should just spend 30 minutes and do it manually (or with a little script).
Have it work with HList input and output
You just need some minor adjustments to the definition of nextstep:
def nextstep[L <: HList](hl: L, data: Map[String, Object])(implicit mapper: Mapper[p1.type, L]): mapper.Out = {
hl map p1
}
I made the exact type of the HList L a type argument, and I required the implicit needed by map (see map's definition).
Then you (or your user) can simply call
nextstep(XS :: XI :: HNil, data)
and they will get a String :: Integer :: HNil as return type. It works as expected for any HList of ColDef[...] (returning an HList of the result).
Have it work with a tuple as input (and output HList)
In order to have it return a tuple instead of an HList, you can define it this way:
import shapeless.ops.hlist.{Tupler, Mapper}
def nextstep[L <: HList, OutL <: HList, Out](hl: L, data: Map[String, Object])(implicit mapper: Mapper.Aux[p1.type, L, OutL], tupler: Tupler.Aux[OutL, Out]): Out = {
tupler(hl map p1)
}
nextstep(XS :: XI :: HNil, data) will then return a (String, Integer), and nextstep will return the rightly typed tuple in the general case.
Tuple as input and output
The final step to accept a tuple of ColDef as input and return a tuple as output looks like:
def nextstep[P, L <: HList, OutL <: HList, Out](c: P, data: Map[String, Object])(implicit gen: Generic.Aux[P, L], mapper: Mapper.Aux[p1.type, L, OutL], tupler: Tupler.Aux[OutL, Out]): Out = {
tupler(gen.to(c) map p1)
}
The logic here is very similar to the functions defined in shapeless.syntax.std.TupleOps: converting the tuple to an HList, processing the HList, convert the output to a tuple.

Find type class instances for Shapeless HList

Say that I have a trait Show[T] such as the one in Scalaz: https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/Show.scala#L9
I also have a Shapeless HList that may look like "1" :: 2 :: 3L :: HNil.
Is there a way to find the Show instance for each element and apply shows such that I end up with "1" :: "2" :: "3L" :: HNil?
If any element were of a type that did not have an implicit Show instance in scope I would want a compile error.
I think that if I build up an HList of the Show instances I should be able to use zipApply to get the HList I want, but I don't know if there is a way to get have Scala infer the HList of Show instances instead of me building it up by hand.
If your goal is to apply the Show instances and you don't otherwise care about building up an HList of them, the easiest approach is probably to use a polymorphic function:
import scalaz._, Scalaz._, shapeless._
val xs = "1" :: 2 :: 3L :: HNil
object show extends Poly1 {
implicit def forShowable[A: Show] = at[A](_.shows)
}
val strings: String :: String :: String :: HNil = xs map show
You could get an HList of the instances by changing the Poly1 a bit:
object showInstance extends Poly1 {
implicit def forShowable[A: Show] = at[A](_ => Show[A])
}
In some cases it can be useful to define your own type class to collect evidence that you've got certain type class instances:
trait AllShowable[L <: HList, S <: HList] {
def instances: S
}
implicit object hnilAllShowable extends AllShowable[HNil, HNil] {
def instances = HNil
}
implicit def hlistAllShowable[H: Show, TL <: HList, TS <: HList](
implicit ts: AllShowable[TL, TS]
) = new AllShowable[H :: TL, Show[H] :: TS] {
def instances = Show[H] :: ts.instances
}
But usually mapping with a polymorphic function that requires the instances will work just fine.

Can Map be performed on a Scala HList

I have done a few implementations of HList now. One based on Daniel Spiewak's High Wizardry in the Land of Scala talk and another based on a post in Apocalisp blog. The goal was to have a heterogenous list of which is not heterogenous in the primary type but rather the higher kind. For example:
val requests = Request[String] :: Request[Int] :: HNil
I would be able to do a map across the list to perform the request and result in a heterogenous list of the higher kind. So:
requests.map(execute)
should equal
String :: Int :: HNil
Sadly all my attempts have resulted in an HList of Any. Here is the code from a recent attempt:
class Request[+Out](o:Out) {
type O = Out
def v:O = o
}
object HList {
trait Func[-Elem,Out] {
type Apply[E <: Elem] <: Out
def apply[N <: Elem](e:N):Apply[N]
}
sealed trait HList[Base] {
type Head <: Base
type Tail <: HList[Base]
type Map[Out,F <: Func[Base,Out]] <: HList[Out]
def head:Head
def tail:Tail
def ::[A <: Base](a:A):HList[Base]
def map[Out,F <: Func[Base,Out]](f:F):Map[Out,F]
}
case class HNil[Base]() extends HList[Base] {
type Head = Nothing
type Tail = Nothing
type Map[Out,F <: Func[Base,Out]] = HNil[Out]
def head = error("Head of an empty HList")
def tail = error("Head of an empty HList")
def ::[A <: Base](a:A) = HCons(a,this)
def map[Out,F <: Func[Base,Out]](f:F) = new HNil[Out]
}
case class HCons[Base,A <: Base,B <: HList[Base]](head: A, tail: B) extends HList[Base] {
type Head = A
type Tail = B
type Map[Out,F <: Func[Base,Out]] = HCons[Out,F#Apply[Head],Tail#Map[Out,F]]
def ::[C <: Base](c:C) = HCons(c,this)
def map[Out,F <: Func[Base,Out]](f:F) =
HCons(f(head),tail.map(f))
}
val :: = HCons
}
object Test extends Application {
import HList._
val HNil = new HNil[Request[_]]
val list = new Request[Int](1) :: new Request[String]("1") :: HNil
val (a :: b :: HNil) = list
val y:Request[String] = b
val results = list.map[Any,Unwrap.type](Unwrap)
val i:Int = results.head
}
import HList._
object Unwrap extends Func[Request[Any],Any] {
type Apply[I <: Request[Any]] = I#O
def apply[N <: Request[Any]](e:N) = null.asInstanceOf[Apply[N]]
}
The other attempt was based on the Apocalisp version which uses fold to create a new HList and again it resulted in a HList of Any types. Any tips would be appreciated.
The HList implementation in shapeless is rich enough to subsume both HList and KList functionality. It provides a map operation which applies a higher-ranked function, possibly with type-specific cases, across it's elements yielding an appropriately typed HList result,
import shapeless.Poly._
import shapeless.HList._
// Define a higher-ranked function from Sets to Options
object choose extends (Set ~> Option) {
def default[T](s : Set[T]) = s.headOption
}
// An HList of Sets
val sets = Set(1) :: Set("foo") :: HNil
// Map our choose function across it ...
val opts = sets map choose
// The resulting value
opts == Option(1) :: Option("foo") :: HNil
Note that although it's the case in the above example there's no requirement that the HList elements share a common outer type constructor, it just has to be the case that the higher-ranked function mapped with has cases for all of the types involved,
// size is a higher-ranked function from values of arbitrary type to a 'size'
// which is defined as 1 by default but which has type specific cases for
// Strings and tuples
object size extends (Id ~> Const[Int]#λ) {
def default[T](t : T) = 1
}
implicit def sizeString = size.λ[String](s => s.length)
implicit def sizeTuple[T, U](implicit st : size.λ[T], su : size.λ[U]) =
size.λ[(T, U)](t => 1+size(t._1)+size(t._2))
size(23) == 1 // Default
size("foo") == 3 // Type specific case for Strings
size((23, "foo")) == 5 // Type specific case for tuples
Now let's map this across an HList,
val l = 23 :: true :: "foo" :: ("bar", "wibble") :: HNil
val ls = l map size
ls == 1 :: 1 :: 3 :: 10 :: HNil
In this case the result type of the function being mapped is constant: it's an Int no matter what the argument type is. Consequently the resulting HList has elements all of the same type, which means that it can usefully be converted to a vanilla list,
ls.toList == List(1, 1, 3, 10)
what you need is a Klist with type constructor Request, and a natural transformation execute: Request ~> Id. All of this is detailed in the marvelous type-level programming series of posts at Apocalisp, in particular:
Natural transformation literals
Klist basics
you can checkout the code for the whole series from Mark Harrah's up repo
In your case, you'll need something like
val reqList = new Request[Int](1) :^: new Request[String]("1") :^: KNil
val exec = new (Request ~> Id) { def apply[T](reqs: Request[T]): T = reqs.execute }
val results = reqList down exec
the down method above is conceptually the same as map for a nat transf M ~> Id; you also have more general map which from a nat transf M ~> N and a Klist of kind M yields a KList of kind N.
Note that you have an example of Map with HList in the recent (October 2016, 5 years after the OP) article "Using shapeless' HLists for extra type safety (in Akka Streams)" from Mikołaj Koziarkiewicz.
//glue for the ParserStageDefs
specs.map(s => Flow[Data].map(s.parser).map(s.processor))
.foreach(broadcast ~> _ ~> merge)
The problem lies in the fact that the type information in our specs list is not preserved. Or rather, not preserved the way we want to - the type of the List elements is ParserStageDef[_ >: Int with String], so the lowest common supertype for our decorator and incrementer.
The above implies that, when mapping between the parser and processor elements, the compiler has no way to provide the actual type T that's used within the given spec.
A solution
Here's where HLists come to the rescue. Because they preserve the complete type information for each element, it's possible to define our flow very similarly to our last attempt.
First, let's replace our list with an HList:
import shapeless.ops.hlist._
import shapeless._
//...
val specs = decorator :: incrementer :: HNil
val specsSize = specs.length.toInt
Now, for the mapping from ParserStageDefs into Flows, we need to take a different approach, as the map for HList requires something called P**oly - a polymorphic function value**.
Here's how one would look like in our case:
import shapeless.PolyDefns.~>
object toFlow extends (ParserStageDef ~> ProcessingFlow) {
override def apply[T](f: ParserStageDef[T]) =
Flow[Data].map(f.parser).map(f.processor)
}
For it to work, we'll also have change ProcessingFlow to type ProcessingFlow[_] = Flow[Data, Data, _], since the polymorphic function above expects a higher-kinded type.
Now, our central statement turns out to be:
//we convert to a List[ProcessingFlow[_]] for simplicity
specs.map(toFlow).toList.foreach(broadcast ~> _ ~> merge)
and we're all set!