Proof about HList structure in shapeless - scala

Sometimes I need a compile-time proof that each element of H <: HList has type T.
It can be represented by this code:
import shapeless._
#annotation.implicitNotFound("Cannot prove that A =:= ${T} forAll A in ${H}")
trait ForAll[H <: HList, T]
object ForAll {
implicit def head[T]: ForAll[T :: HNil, T] = new ForAll[T :: HNil, T] {}
implicit def tail[T, HT <: HList](implicit ttail: ForAll[HT, T]): ForAll[T :: HT, T] = new ForAll[T :: HT, T] {}
}
// an example
def chain[H <: HList, A](hlist: H)(a: A)(implicit allAreFunctions: ForAll[H, (A => A)]): A = {
def iter(h: HList, ax: A): A = h match {
case HNil => ax
case (f: (A => A)) :: tail => iter(tail, f(ax))
}
iter(hlist, a)
}
val hlist1 = ((_: Int) + 1) :: ((_: Int) * 1) :: ((_: Int) - 2) :: HNil
val hlist2 = ((_: Int).toString) :: ((_: Int) + 1) :: ((_: Int) - 2) ::HNil
chain(hlist1)(1)
// chain(hlist2)(1) doesn't compile
I can ignore compiler warning about possible match error because ForAll instance proves that the code is correct.
Does shapeless implement something like this (or maybe there are a better alternative)?

ForAll is just shapeless.ops.hlist.LeftReducer for properly defined Poly. For example
import shapeless.{::, HNil, Poly2}
import shapeless.ops.hlist.LeftReducer
object myPoly extends Poly2 {
implicit def `case`[A]: Case.Aux[A, A, A] = at((x, y) => x)
}
implicitly[LeftReducer.Aux[Int :: Int :: Int :: Int :: HNil, myPoly.type, Int]]//compiles
implicitly[LeftReducer[Int :: Int :: Int :: Int :: HNil, myPoly.type]]//compiles
// implicitly[LeftReducer.Aux[Int :: Int :: String :: Int :: HNil, myPoly.type, Int]]//doesn't compile
// implicitly[LeftReducer[Int :: Int :: String :: Int :: HNil, myPoly.type]]//doesn't compile
LeftReducer[Int :: Int :: Int :: Int :: HNil, myPoly.type].apply(1 :: 2 :: 3 :: 4 :: HNil)//1
// LeftReducer[Int :: Int :: String :: Int :: HNil, myPoly.type].apply(1 :: 2 :: "a" :: 4 :: HNil)//doesn't compile
(1 :: 2 :: 3 :: 4 :: HNil).reduceLeft(myPoly)//1
// (1 :: 2 :: "a" :: 4 :: HNil).reduceLeft(myPoly)//doesn't compile

Related

How to concatenate function arguments and return values using shapeless

I was trying to define compose method below:
import shapeless.{::, HList, HNil}
def compose[A1 <: HList, R1, A2 <: HList, R2](f: A1 => R1, g: A2 => R2): R1 :: R2 :: HNil = ???
val f = (xs: Int :: String :: HNil) => xs.select[Int].toString + xs.select[String]
val g = (xs: Boolean :: HNil) => xs.select[Boolean]
// expected output:
// Int :: String :: Boolean :: HNil => String :: Boolean :: HNil
compose(f, g)
This is my incomplete code and I have no idea how to get A1 and A2 from args. Can anyone help, please?
def compose[A1 <: HList, R1, A2 <: HList, R2](f: A1 => R1, g: A2 => R2)(implicit p: Prepend[A2, A1]) =
(args: p.Out) => {
val a1: A1 = ???
val a2: A2 = ???
f(a1) :: f(a2) :: HNil
}
I was able to implement a working solution with Split:
def compose[A1 <: HList, R1, A2 <: HList, R2, O <: HList, N <: Nat](f: A1 => R1, g: A2 => R2)(
implicit split: Split.Aux[O, N, A1, A2]
): O => R1 :: R2 :: HNil = {
(a: O) =>
val (a1, a2) = split.apply(a)
f(a1) :: g(a2) :: HNil //I assumed you meant calling g(a2) here
}
val f = (xs: Int :: String :: HNil) => xs.select[Int].toString + xs.select[String]
val g = (xs: Boolean :: HNil) => xs.select[Boolean]
val r: Int :: String :: Boolean :: HNil => String :: Boolean :: HNil = compose(f, g)
println(r.apply(1 :: "hello" :: true :: HNil)) // 1hello :: true :: HNil

How to apply a separate function to each index of a known Shapelss HList

I know the type of an HList and I would like to apply a specific function to each of the specific items in the HList. For example.
I know the type of the HList, in this case String :: Int :: String :: HNil
val values = "Hello World" :: 5 :: "Goodbye World" :: HNil
So I create a function from type => Int for each item.
val funcs =
((_: String) => 1) ::
((_: Int) => 2) ::
((_: String) => 3) ::
HNil
I want to apply function 1 to value 1, function 2 to value 2, etc for each item in the list. This is the best guess at what the function signature might look. V
// V are the Values of type String :: Int :: String :: HNil
// F are the Functions of type (String => Int) :: (Int => Int) :: (String => Int) :: HNil
// R is the Int Results of type Int :: Int :: Int :: HNil
def applyFunction[V <: HList, F <: HList, R <: HList](v: V, f: F)(
implicit ev: V =:= F,
utcc: UnaryTCConstraint[F, Function[*, Int]]
): R
Now I'm struggling with how to actually implement this function or if maybe Shapeless already has something that does this already.
Using this SO answer as an inspiration, I came up with the following:
import shapeless._
val values = "Hello World" :: 5 :: "Goodbye World" :: HNil
val funcs = ((_: String) => 1) ::
((_: Int) => 2) ::
((_: String) => 3) :: HNil
object applyTo extends Poly1 {
implicit def caseInt = at[(Int=>Int,Int)] {case (f,v) => f(v)}
implicit def caseStr = at[(String=>Int,String)]{case (f,v) => f(v)}
}
funcs.zip(values).map(applyTo)
//1 :: 2 :: 3 :: HNil: Int :: Int :: Int :: shapeless.HNil
You can use standard type class shapeless.ops.hlist.ZipApply
funcs.zipApply(values) // 1 :: 2 :: 3 :: HNil

Transform a HList of Eithers to an Either of a HList

I want to define a function that accepts a HList whose elements are such that, for each element t, there is a type T such that t: Either[String, T]. The function, which we will call validate, should have the following behaviour:
If all elements of the parameter are Right, return Right of the result of mapping the parameter with right-projection.
Otherwise, return a Left[List[String]], where the list contains the left-projection for each Left in the parameter.
Examples:
validate (Right (42) :: Right (3.14) :: Right (false) :: HNil)
>> Right (42 :: 3.14 :: false :: HNil)
validate (Right (42) :: Left ("qwerty") :: Left ("uiop") :: HNil)
>> Left (List ("qwerty", "uiop"))
An example use case:
case class Result (foo: Foo, bar: Bar, baz: Baz, qux: Qux)
def getFoo: Either[String, Foo] = ???
def getBar: Either[String, Bar] = ???
def getBaz: Either[String, Baz] = ???
def getQux: Either[String, Qux] = ???
def createResult: Either[String, Result] = {
validate (getFoo :: getBar :: getBaz :: getQux :: HNil) match {
case Right (foo :: bar :: baz :: qux :: HNil) => Right (Result (foo, bar, baz, qux))
case Left (errors) => Left ("The following errors occurred:\n" + errors.mkString ("\n"))
}
}
I'll assume we have some test data like this throughout this answer:
scala> import shapeless.{::, HNil}
import shapeless.{$colon$colon, HNil}
scala> type In = Either[String, Int] :: Either[String, String] :: HNil
defined type alias In
scala> val good: In = Right(123) :: Right("abc") :: HNil
good: In = Right(123) :: Right(abc) :: HNil
scala> val bad: In = Left("error 1") :: Left("error 2") :: HNil
bad: In = Left(error 1) :: Left(error 2) :: HNil
Using a custom type class
There are many ways you could do this. I'd probably use a custom type class that highlights the way instances are built up inductively:
import shapeless.HList
trait Sequence[L <: HList] {
type E
type Out <: HList
def apply(l: L): Either[List[E], Out]
}
object Sequence {
type Aux[L <: HList, E0, Out0 <: HList] = Sequence[L] { type E = E0; type Out = Out0 }
implicit def hnilSequence[E0]: Aux[HNil, E0, HNil] = new Sequence[HNil] {
type E = E0
type Out = HNil
def apply(l: HNil): Either[List[E], HNil] = Right(l)
}
implicit def hconsSequence[H, T <: HList, E0](implicit
ts: Sequence[T] { type E = E0 }
): Aux[Either[E0, H] :: T, E0, H :: ts.Out] = new Sequence[Either[E0, H] :: T] {
type E = E0
type Out = H :: ts.Out
def apply(l: Either[E0, H] :: T): Either[List[E0], H :: ts.Out] =
(l.head, ts(l.tail)) match {
case (Right(h), Right(t)) => Right(h :: t)
case (Left(eh), Left(et)) => Left(eh :: et)
case (Left(eh), _) => Left(List(eh))
case (_, Left(et)) => Left(et)
}
}
}
Then you can write validate like this:
def validate[L <: HList](l: L)(implicit s: Sequence[L]): Either[List[s.E], s.Out] = s(l)
And use it like this:
scala> validate(good)
res0: scala.util.Either[List[String],Int :: String :: shapeless.HNil] = Right(123 :: abc :: HNil)
scala> validate(bad)
res1: scala.util.Either[List[String],Int :: String :: shapeless.HNil] = Left(List(error 1, error 2))
Note that the static types come out right.
Using a right fold
You could also do it a little more concisely by folding with a Poly2.
import shapeless.Poly2
object combine extends Poly2 {
implicit def eitherCase[H, T, E, OutT <: HList]:
Case.Aux[Either[E, H], Either[List[E], OutT], Either[List[E], H :: OutT]] = at {
case (Right(h), Right(t)) => Right(h :: t)
case (Left(eh), Left(et)) => Left(eh :: et)
case (Left(eh), _) => Left(List(eh))
case (_, Left(et)) => Left(et)
}
}
And then:
scala> good.foldRight(Right(HNil): Either[List[String], HNil])(combine)
res2: scala.util.Either[List[String],Int :: String :: shapeless.HNil] = Right(123 :: abc :: HNil)
scala> bad.foldRight(Right(HNil): Either[List[String], HNil])(combine)
res3: scala.util.Either[List[String],Int :: String :: shapeless.HNil] = Left(List(error 1, error 2))
I guess this is probably the "right" answer, assuming you want to stick to Shapeless alone. The Poly2 approach just relies on some weird details of implicit resolution (we couldn't define combine as a val, for example) that I personally don't really like.
Using Kittens's sequence
Lastly you could use the Kittens library, which supports sequencing and traversing hlists:
scala> import cats.instances.all._, cats.sequence._
import cats.instances.all._
import cats.sequence._
scala> good.sequence
res4: scala.util.Either[String,Int :: String :: shapeless.HNil] = Right(123 :: abc :: HNil)
scala> bad.sequence
res5: scala.util.Either[String,Int :: String :: shapeless.HNil] = Left(error 1)
Note that this doesn't accumulate errors, though.
If you wanted the most complete possible Typelevel experience I guess you could add a parSequence operation to Kittens that would accumulate errors for an hlist of eithers via the Parallel instance mapping them to Validated (see my blog post here for more detail about how this works). Kittens doesn't currently include this, though.
Update: parallel sequencing
If you want parSequence, it's not actually that much of a nightmare to write it yourself:
import shapeless.HList, shapeless.poly.~>, shapeless.ops.hlist.{Comapped, NatTRel}
import cats.Parallel, cats.instances.all._, cats.sequence.Sequencer
def parSequence[L <: HList, M[_], P[_], PL <: HList, Out](l: L)(implicit
cmp: Comapped[L, M],
par: Parallel.Aux[M, P],
ntr: NatTRel[L, M, PL, P],
seq: Sequencer.Aux[PL, P, Out]
): M[Out] = {
val nt = new (M ~> P) {
def apply[A](a: M[A]): P[A] = par.parallel(a)
}
par.sequential(seq(ntr.map(nt, l)))
}
And then:
scala> parSequence(good)
res0: Either[String,Int :: String :: shapeless.HNil] = Right(123 :: abc :: HNil)
scala> parSequence(bad)
res1: Either[String,Int :: String :: shapeless.HNil] = Left(error 1error 2)
Note that this does accumulate errors, but by concatenating the strings. The Cats-idiomatic way to accumulate errors in a list would look like this:
scala> import cats.syntax.all._
import cats.syntax.all._
scala> val good = 123.rightNel[String] :: "abc".rightNel[String] :: HNil
good: Either[cats.data.NonEmptyList[String],Int] :: Either[cats.data.NonEmptyList[String],String] :: shapeless.HNil = Right(123) :: Right(abc) :: HNil
scala> val bad = "error 1".leftNel[String] :: "error 2".leftNel[Int] :: HNil
bad: Either[cats.data.NonEmptyList[String],String] :: Either[cats.data.NonEmptyList[String],Int] :: shapeless.HNil = Left(NonEmptyList(error 1)) :: Left(NonEmptyList(error 2)) :: HNil
scala> parSequence(good)
res3: Either[cats.data.NonEmptyList[String],Int :: String :: shapeless.HNil] = Right(123 :: abc :: HNil)
scala> parSequence(bad)
res4: Either[cats.data.NonEmptyList[String],String :: Int :: shapeless.HNil] = Left(NonEmptyList(error 1, error 2))
It'd probably be worth opening a PR to add something like this to Kittens.
I managed to arrive at a solution essentially identical to Travis Brown's right-fold solution, with a few additions:
class Validate[E] {
def apply[L <: HList] (hlist: L) (implicit folder: RightFolder[L, Either[List[E], HNil], combine.type]) =
hlist.foldRight (Right (HNil) : Either[List[E], HNil]) (combine)
}
object combine extends Poly2 {
implicit def combine[E, H, T <: HList]
: ProductCase.Aux[Either[E, H] :: Either[List[E], T] :: HNil, Either[List[E], H :: T]] = use {
(elem: Either[E, H], result: Either[List[E], T]) => (elem, result) match {
case (Left (error), Left (errors)) => Left (error :: errors)
case (Left (error), Right (_)) => Left (error :: Nil)
case (Right (_), Left (errors)) => Left (errors)
case (Right (value), Right (values)) => Right (value :: values)
}
}
}
def validate[E] = new Validate[E]
This allows the left type to vary, and allows the syntax:
validate[String] (getFoo :: getBar :: getBaz :: getQux :: HNil) match {
case Right (foo :: bar :: baz :: qux :: HNil) => ???
case Left (errors) => ???
}
Admittedly this is the first time that I have used Poly. It blew my mind to see that this actually worked. Infuriatingly, the static analysis provided by my IDE (IntelliJ) is not clever enough to infer the types of the terms in the match cases.

Not able to build a shapeless HList recursively

I wrote this code
object HList1Test extends App {
import shapeless.{HList, HNil}
import shapeless.syntax._
import shapeless.ops._
def myFunc(list: List[Int], result: HList = HNil) : HList = {
def double (i: Int) : HList = if (i % 2 == 0) 2 * i :: HNil else { (2 * i).toString :: HNil}
list match {
case h :: t => {
myFunc(t, double(h) ::: result)
}
case Nil => result
}
}
println(myFunc(List(1, 2, 3)))
}
But I get error
Error:(16, 33) could not find implicit value for parameter prepend: shapeless.ops.hlist.Prepend[shapeless.HList,shapeless.HList]
myFunc(t, double(h) ::: hlist)
Error:(16, 33) not enough arguments for method :::: (implicit prepend: shapeless.ops.hlist.Prepend[shapeless.HList,shapeless.HList])prepend.Out.
Unspecified value parameter prepend.
myFunc(t, double(h) ::: hlist)
My end goal was to have a HList with "2" :: 4 :: "6" :: HNil
Saying that 2 * i :: HNil and (2 * i).toString :: HNil have type just HList (rather than Int :: HNil and String :: HNil correspondingly) is too rough.
Since double actually returns values of types Int :: HNil or String :: HNil depending on value of its input argument, double is a Poly. So is myFunc.
object double extends Poly1 {
implicit def evenCase[N <: Nat](implicit
mod: Mod.Aux[N, _2, _0],
toInt: ToInt[N]): Case.Aux[N, Int :: HNil] =
at(n => 2 * toInt() :: HNil)
implicit def oddCase[N <: Nat](implicit
mod: Mod.Aux[N, _2, _1],
toInt: ToInt[N]): Case.Aux[N, String :: HNil] =
at(n => (2 * toInt()).toString :: HNil)
}
object myFunc extends Poly2 {
implicit def hNilCase[R <: HList]: Case.Aux[HNil, R, R] = at((_, r) => r)
implicit def hConsCase[H <: Nat, T <: HList,
dblH <: HList, R <: HList,
P <: HList, Out <: HList](implicit
dbl: double.Case.Aux[H, dblH],
prepend: Prepend.Aux[dblH, R, P],
myF: myFunc.Case.Aux[T, P, Out]): Case.Aux[H :: T, R, Out] =
at { case (h :: t, r) => myFunc(t, double(h) ::: r) }
}
println(myFunc(_1 :: _2 :: _3 :: HNil, HNil)) //"6" :: 4 :: "2" :: HNil

Weird behavior trying to convert case classes to heterogeneous lists recursively with Shapeless

I stayed up way too late last night trying to figure out this Shapeless issue and I'm afraid it's going to eat my evening if I don't get it off my chest, so here goes.
In this minimized version I'm just defining a type class that will recursively convert case classes into heterogeneous lists:
import shapeless._
trait DeepHLister[R <: HList] extends DepFn1[R] { type Out <: HList }
trait LowPriorityDeepHLister {
type Aux[R <: HList, Out0 <: HList] = DeepHLister[R] { type Out = Out0 }
implicit def headNotCaseClassDeepHLister[H, T <: HList](implicit
dht: DeepHLister[T]
): Aux[H :: T, H :: dht.Out] = new DeepHLister[H :: T] {
type Out = H :: dht.Out
def apply(r: H :: T) = r.head :: dht(r.tail)
}
}
object DeepHLister extends LowPriorityDeepHLister {
implicit object hnilDeepHLister extends DeepHLister[HNil] {
type Out = HNil
def apply(r: HNil) = HNil
}
implicit def headCaseClassDeepHLister[H, R <: HList, T <: HList](implicit
gen: Generic.Aux[H, R],
dhh: DeepHLister[R],
dht: DeepHLister[T]
): Aux[H :: T, dhh.Out :: dht.Out] = new DeepHLister[H :: T] {
type Out = dhh.Out :: dht.Out
def apply(r: H :: T) = dhh(gen.to(r.head)) :: dht(r.tail)
}
def apply[R <: HList](implicit dh: DeepHLister[R]): Aux[R, dh.Out] = dh
}
Let's try it out! First we need some case classes:
case class A(x: Int, y: String)
case class B(x: A, y: A)
case class C(b: B, a: A)
case class D(a: A, b: B)
And then (note that I've cleaned up the type syntax for the sake of this not being a totally unreadable mess):
scala> DeepHLister[A :: HNil]
res0: DeepHLister[A :: HNil]{
type Out = (Int :: String :: HNil) :: HNil
} = DeepHLister$$anon$2#634bf0bf
scala> DeepHLister[B :: HNil]
res1: DeepHLister[B :: HNil] {
type Out = (
(Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil
) :: HNil
} = DeepHLister$$anon$2#69d6b3e1
scala> DeepHLister[C :: HNil]
res2: DeepHLister[C :: HNil] {
type Out = (
((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) ::
(Int :: String :: HNil) ::
HNil
) :: HNil
} = DeepHLister$$anon$2#4d062faa
So far so good. But then:
scala> DeepHLister[D :: HNil]
res3: DeepHLister[D :: HNil] {
type Out = ((Int :: String :: HNil) :: B :: HNil) :: HNil
} = DeepHLister$$anon$2#5b2ab49a
The B didn't get converted. If we turn on -Xlog-implicits this is the last message:
<console>:25: this.DeepHLister.headCaseClassDeepHLister is not a valid implicit value for DeepHLister[shapeless.::[B,shapeless.HNil]] because:
hasMatchingSymbol reported error: diverging implicit expansion for type DeepHLister[this.Repr]
starting with method headNotCaseClassDeepHLister in trait LowPriorityDeepHLister
DeepHLister[D :: HNil]
^
Which doesn't make sense to me—headCaseClassDeepHLister should be able to generate DeepHLister[B :: HNil] just fine, and it does if you ask it directly.
This happens on both 2.10.4 and 2.11.2, and with both the 2.0.0 release and master. I'm pretty sure this has to be a bug, but I'm not ruling out the possibility that I'm doing something wrong. Has anyone seen anything like this before? Is there something wrong with my logic or some restriction on Generic I'm missing?
Okay, thanks for listening—maybe now I can go read a book or something.
This now works more or less as written using recent shapeless-2.1.0-SNAPSHOT builds, and a close relative of the sample in this question has been added there as an example.
The problem with the original is that each expansion of a Generic introduces a new HList type into the implicit resolution of the DeepHLister type class instances and, in principle, could produce an HList type that is related to but more complex than some type seen previously during the same resolution. This condition trips the divergence checker and aborts the resolution process.
The exact details of why this happens for D but not for C is lurking in the details of the implementation of Scala's typechecker but, to a rough approximation, the differentiator is that during the resolution for C we see the B (larger) before the A (smaller) so the divergence checker is happy that our types are converging; conversely during the resolution for D we see the A (smaller) before the B (larger) so the divergence checker (conservatively) bails.
The fix for this in shapeless 2.1.0 is the recently enhanced Lazy type constructor and associated implicit macro infrastructure. This allows much more user control over divergence and supports the use of implicit resolution to construct the recursive implicit values which are crucial to the ability to automatically derive type class instances for recursive types. Many examples of this can be found in the shapeless code base, in particular the reworked type class derivation infrastructure and Scrap Your Boilerplate implementation, which no longer require dedicated macro support, but are implemented entirely in terms of the Generic and Lazy primitives. Various applications of these mechanisms can be found in the shapeless examples sub-project.
I took a slightly different approach.
trait CaseClassToHList[X] {
type Out <: HList
}
trait LowerPriorityCaseClassToHList {
implicit def caseClass[X](implicit gen: Generic[X]): CaseClassToHList[X] {
type Out = generic.Repr
} = null
}
object CaseClassToHList extends LowerPriorityCaseClassToHList {
type Aux[X, R <: HList] = CaseClassToHList[X] { type Out = R }
implicit def caseClassWithCaseClasses[X, R <: HList](
implicit toHList: CaseClassToHList.Aux[X, R],
nested: DeepHLister[R]): CaseClassToHList[X] {
type Out = nested.Out
} = null
}
trait DeepHLister[R <: HList] {
type Out <: HList
}
object DeepHLister {
implicit def hnil: DeepHLister[HNil] { type Out = HNil } = null
implicit def caseClassAtHead[H, T <: HList](
implicit head: CaseClassToHList[H],
tail: DeepHLister[T]): DeepHLister[H :: T] {
type Out = head.Out :: tail.Out
} = null
def apply[X <: HList](implicit d: DeepHLister[X]): d.type = null
}
Tested with the following code:
case class A(x: Int, y: String)
case class B(x: A, y: A)
case class C(b: B, a: A)
case class D(a: A, b: B)
object Test {
val z = DeepHLister[HNil]
val typedZ: DeepHLister[HNil] {
type Out = HNil
} = z
val a = DeepHLister[A :: HNil]
val typedA: DeepHLister[A :: HNil] {
type Out = (Int :: String :: HNil) :: HNil
} = a
val b = DeepHLister[B :: HNil]
val typedB: DeepHLister[B :: HNil] {
type Out = ((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil
} = b
val c = DeepHLister[C :: HNil]
val typedC: DeepHLister[C :: HNil] {
type Out = (((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil
} = c
val d = DeepHLister[D :: HNil]
val typedD: DeepHLister[D :: HNil] {
type Out = ((Int :: String :: HNil) :: ((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil) :: HNil
} = d
}