Unzip a query of tuple in Slick+shapeless - scala

I have a Slick (grouping) query returning a tuple of integers which should be then combined using one of SQL aggregation functions. The size of the tuple is not dynamic but I would like to be able to define it in a single place with possibility to change it later. I use shapeless elsewhere in my project so it is natural to use it here as well.
So, let's say we have a tuple type type Agg = (Rep[Int], Rep[Int], Rep[Int]) and a query with type Query[Agg, _, Seq]
I can then define poly's for aggregation like this:
object sum extends (Query[Rep[Int], Int, Seq] -> Rep[Int])(_.sum.getOrElse(0))
object sumAgg extends(Query[Agg, _, Seq] -> Agg)(q => (q.map(_._1), q.map(_._2), q.map(_._3)).map(sum))
But I can't find a way to get rid of explicit tuple unzipping in sumAgg poly. How can I transform a Query of tuple of Int's (aka Agg) into a tuple of queries of Int's?
Let's even simplify this. Supposing I have
val hlist = 1 :: 2 :: 3 :: HNil
val tuple = (4, 5, 6)
hlist.zipWithIndex.map(m)
What would be the definition of m to produce, say, a hlist of (1*4) :: (2*5) :: (3*6) :: HNil? I know I could directly zip hlist with tuple but in this scenario I think I do need to pick tuple elements one by one by their positions.

Try to replace sumAgg with type classes.
import shapeless.{::, HList, HNil, Nat, Poly1, Succ}
import shapeless.nat._
import shapeless.poly.->
import shapeless.syntax.std.tuple._
import shapeless.ops.hlist.Tupler
import shapeless.ops.tuple.{At, Length, Mapper}
import slick.lifted.{Query, Rep, Shape}
import slick.jdbc.PostgresProfile.api._
trait MkHList[Agg <: Product, N <: Nat] {
type Out <: HList
def apply(q: Query[Agg, _, Seq]): Out
}
object MkHList {
type Aux[Agg <: Product, N <: Nat, Out0 <: HList] = MkHList[Agg, N] { type Out = Out0 }
def instance[Agg <: Product, N <: Nat, Out0 <: HList](f: Query[Agg, _, Seq] => Out0): Aux[Agg, N, Out0] = new MkHList[Agg, N] {
override type Out = Out0
override def apply(q: Query[Agg, _, Seq]): Out = f(q)
}
implicit def zero[Agg <: Product]: Aux[Agg, _0, HNil] = instance(_ => HNil)
implicit def succ[Agg <: Product, N <: Nat, A](implicit
tailMkHList: MkHList[Agg, N],
at: At.Aux[Agg, N, Rep[A]],
shape: Shape[_ <: FlatShapeLevel, Rep[A], A, Rep[A]]
): Aux[Agg, Succ[N], Query[Rep[A], A, Seq] :: tailMkHList.Out] =
instance(q => q.map(_.at[N]) :: tailMkHList(q))
}
trait SumAgg[Agg <: Product] {
def apply(q: Query[Agg, _, Seq]): Agg
}
object SumAgg {
implicit def mkSumAgg[Agg <: Product, N <: Nat, L <: HList, Tpl <: Product](implicit
length: Length.Aux[Agg, N],
mkHList: MkHList.Aux[Agg, N, L],
tupler: Tupler.Aux[L, Tpl],
mapper: Mapper.Aux[Tpl, sum.type, Agg]
): SumAgg[Agg] = q => tupler(mkHList(q)).map(sum)
}
def sumAgg[Agg <: Product](q: Query[Agg, _, Seq])(implicit sumAggInst: SumAgg[Agg]): Agg = sumAggInst(q)
type Agg = (Rep[Int], Rep[Int], Rep[Int])
sumAgg(??? : Query[Agg, _, Seq]): Agg

Answering your simplified version:
import shapeless._
import syntax.std.tuple._ // brings implicits for tuple.productElements
// defines your polymorphic mapper
object MulPoly extends Poly1 {
// you're going to need one case for each pair of types that you might face.
// E.g. (Int, Int), (Int, String), (String, String), etc.
implicit val intPairCase: Case.Aux[(Int, Int), Int] = at({ case (a, b) => a * b })
}
val hlist = 1 :: 2 :: 3 :: HNil
val tuple = (4, 5, 6)
val tupleHList = tuple.productElements
>>> tupleHList: Int :: Int :: Int :: shapeless.HNil = 4 :: 5 :: 6 :: HNil
hlist
.zip(tupleHList) // intermediate result here: (1, 4) :: (2, 5) :: (3, 6) :: HNil
.map(MulPoly) // map MulPoly over the HList
>>> res0: Int :: Int :: Int :: shapeless.HNil = 4 :: 10 :: 18 :: HNil
So, essentially, your m is a Poly1, that can map on 2-tuples of types that constitute your hlist and a tuple.
You might want to consult with Type Astronaut's Guide to Shapeless, Section 7 about ploymorphic mapping, and this stackoverflow question about how to get an hlist out of any tuple.

Related

Zip generic HList with static Nat HList

I'm searching a way to zip two HList together. The first one is generated from a case class converted in its generic representation, and the second one is defined manually as an HList of Nat.
As a result, I expect a tuple (or 2-members HList) with one field from the case class, and the Nat associated.
The goal is to create a "customizable" ZipWithIndex.
def derive[A, I <: HList, R <: HList, Z <: HList](implicit
gen: Generic.Aux[A, R],
zipper: Zip.Aux[R, I, Z],
enc: Lazy[Encoder[Z]])(a: A): Deriver[A] = {
val genRepr = gen.to(A)
val zipped = zip(genRepr :: ??? :: HNil)
enc.value(zipped)
}
case class Foo(a: String, b: String, c: String)
derive[Foo, Nat._1 :: Nat._3 :: Nat.7 :: HNil]
The result will have to match an encoderTuple[H, N <: Nat, T <: HList]: Encoder[(H, N) :: T] or and encoderHList[H, N <: Nat, T <: HList]: Encoder[(H::N::HNil) :: T].
The goal is to create a "customizable" ZipWithIndex.
I guess standard shapeless.ops.hlist.Zip should be enough for that:
trait Encoder[Z] {
type Out
def apply(z: Z): Out
}
object Encoder {
type Aux[Z, Out0] = Encoder[Z] { type Out = Out0 }
// implicits
}
trait Deriver[A]
object Deriver {
// implicits
}
def derive[A, I <: HList, R <: HList, Z <: HList](a: A, i: I)(implicit
gen: Generic.Aux[A, R],
zipper: Zip.Aux[R :: I :: HNil, Z],
enc: Lazy[Encoder.Aux[Z, Deriver[A]]]): Deriver[A] = {
val genRepr: R = gen.to(a)
val zipped: Z = zipper(genRepr :: i :: HNil)
enc.value(zipped)
}
case class Foo(a: String, b: String, c: String)
// derive[Foo, Nat._1 :: Nat._3 :: Nat._7 :: HNil, ???, ???]
derive(Foo("aaa", "bbb", "ccc"), Nat._1 :: Nat._3 :: Nat._7 :: HNil)

Avoid losing type info in return value

I'm trying to find a way to avoid losing type information in the return value for my method.
I have the following:
val defs0 = Default.mkDefault[Person, Some[String] :: Some[Int] :: HNil](Some("odd") :: Some(42) :: HNil)
Using IntelliJs "Add type annotation" gives the type:
Default.Aux[Person, ::[Some[String], ::[Some[Int], HNil]]]
This is fine, except I dont want to specify the fields of Person when I call mkDefault. So I created this:
object MkDefault {
object toSome extends Poly1 {
implicit def default[P] = at[P](Some(_))
}
def apply[P, L <: HList, D <: HList]
(p: P)
(implicit
lg: LabelledGeneric.Aux[P, L],
mpr: Mapper.Aux[toSome.type, L, D]
): Default.Aux[P, D] =
Default.mkDefault[P, D](mpr(lg.to(p)))
}
Now I can do:
val defs1 = MkDefault(Person("odd", 42))
Which is good, except the inferred type from IntellJ looks like this:
Default.Aux[Person, HNil]
How can I make the inferred type of defs1 equal to the inferred type of defs0?
*without having to specify the fields of class Person
Use mpr.Out instead of D as an output type:
object MkDefault {
object toSome extends Poly1 {
implicit def default[P] = at[P](Some(_))
}
def apply[P, L <: HList, D <: HList]
(p: P)
(implicit
lg: LabelledGeneric.Aux[P, L],
mpr: Mapper.Aux[toSome.type, L, D]
): Default.Aux[P, mpr.Out] =
Default.mkDefault[P, D](mpr(lg.to(p)))
}
Example:
scala> val defs1 = MkDefault(Person("odd", 42))
defs1: shapeless.Default[Person]{type Out = shapeless.::[Some[String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("name")],String]],shapeless.::[Some[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("age")],Int]],shapeless.HNil]]} = shapeless.Default$$anon$1#1f6a8bb8
P.S.
If you don't mind Option instead of Some, you can also use AsOptions
val opt = Default.AsOptions[Person]
val def3 = Default.mkDefault[Person, opt.Out](Some("aaa") :: Some(5) :: HNil)
Check:
scala> implicitly[def3.Out =:= ::[Option[String], ::[Option[Int], HNil]]]
res12: =:=[def3.Out,shapeless.::[Option[String],shapeless.::[Option[Int],shapeless.HNil]]] = <function1>

Missing Sized.unapply

In object Sized (in "shapeless/sized.scala") there is unapplySeq, which unfortunately doesn't provide static checking. For example code below will fail at runtime with MatchError:
Sized(1, 2, 3) match { case Sized(x, y) => ":(" }
It would be better if there was unapply method instead, returning Option of tuple, and concrete shape of tuple was constructed according to size of Sized instance. For example:
Sized(1) => x
Sized(1, 2) => (x, y)
Sized(1, 2, 3) => (x, y, z)
In that case previous code snippet would fail to compile with constructor cannot be instantiated to expected type.
Please help me implement unapply for object Sized. Is this method already implemented anywhere?
Thanks in advance!
This is definitely possible (at least for Sized where N is less than 23), but the only approach I can think of (barring macros, etc.) is kind of messy. First we need a type class that'll help us convert sized collections to HLists:
import shapeless._, Nat._0
import scala.collection.generic.IsTraversableLike
trait SizedToHList[R, N <: Nat] extends DepFn1[Sized[R, N]] {
type Out <: HList
}
object SizedToHList {
type Aux[R, N <: Nat, Out0 <: HList] = SizedToHList[R, N] { type Out = Out0 }
implicit def emptySized[R]: Aux[R, Nat._0, HNil] = new SizedToHList[R, _0] {
type Out = HNil
def apply(s: Sized[R, _0]) = HNil
}
implicit def otherSized[R, M <: Nat, T <: HList](implicit
sth: Aux[R, M, T],
itl: IsTraversableLike[R]
): Aux[R, Succ[M], itl.A :: T] = new SizedToHList[R, Succ[M]] {
type Out = itl.A :: T
def apply(s: Sized[R, Succ[M]]) = s.head :: sth(s.tail)
}
def apply[R, N <: Nat](implicit sth: SizedToHList[R, N]): Aux[R, N, sth.Out] =
sth
def toHList[R, N <: Nat](s: Sized[R, N])(implicit
sth: SizedToHList[R, N]
): sth.Out = sth(s)
}
And then we can define an extractor object that uses this conversion:
import ops.hlist.Tupler
object SafeSized {
def unapply[R, N <: Nat, L <: HList, T <: Product](s: Sized[R, N])(implicit
itl: IsTraversableLike[R],
sth: SizedToHList.Aux[R, N, L],
tupler: Tupler.Aux[L, T]
): Option[T] = Some(sth(s).tupled)
}
And then:
scala> val SafeSized(x, y, z) = Sized(1, 2, 3)
x: Int = 1
y: Int = 2
z: Int = 3
But:
scala> val SafeSized(x, y) = Sized(1, 2, 3)
<console>:18: error: wrong number of arguments for object SafeSized
val SafeSized(x, y) = Sized(1, 2, 3)
^
<console>:18: error: recursive value x$1 needs type
val SafeSized(x, y) = Sized(1, 2, 3)
^
As desired.

Creating an HList of all pairs from two HLists

I'm using shapeless in Scala, and I'd like to write a function allPairs that will take two HLists and return an HList of all pairs of elements. For example:
import shapeless._
val list1 = 1 :: "one" :: HNil
val list2 = 2 :: "two" :: HNil
// Has value (1, 2) :: (1, "two") :: ("one", 2) :: ("one", "two") :: HNil
val list3 = allPairs(list1, list2)
Any idea how to do this?
Also, I'd like to emphasize I'm looking for a function, not an inlined block of code.
You can't use a for-comprehension or a combination of map and flatMap with function literals here (as the other answers suggest), since these methods on HList require higher rank functions. If you just have two statically typed lists, this is easy:
import shapeless._
val xs = 1 :: 'b :: 'c' :: HNil
val ys = 4.0 :: "e" :: HNil
object eachFirst extends Poly1 {
implicit def default[A] = at[A] { a =>
object second extends Poly1 { implicit def default[B] = at[B](a -> _) }
ys map second
}
}
val cartesianProductXsYs = xs flatMap eachFirst
Which gives us the following (appropriately typed):
(1,4.0) :: (1,e) :: ('b,4.0) :: ('b,e) :: (c,4.0) :: (c,e) :: HNil
Writing a method that will do this with HList arguments is trickier. Here's a quick example of how it can be done (with some slightly more general machinery).
I'll start by noting that we can think of finding the Cartesian product of two ordinary lists as "lifting" a function that takes two arguments and returns them as a tuple into the applicative functor for lists. For example, you can write the following in Haskell:
import Control.Applicative (liftA2)
cartesianProd :: [a] -> [b] -> [(a, b)]
cartesianProd = liftA2 (,)
We can write a polymorphic binary function that corresponds to (,) here:
import shapeless._
object tuple extends Poly2 {
implicit def whatever[A, B] = at[A, B] { case (a, b) => (a, b) }
}
And define our example lists again for completeness:
val xs = 1 :: 'b :: 'c' :: HNil
val ys = 4.0 :: "e" :: HNil
Now we'll work toward a method named liftA2 that will allow us to write the following:
liftA2(tuple)(xs, ys)
And get the correct result. The name liftA2 is a little misleading, since we don't really have an applicative functor instance, and since it's not generic—I'm working on the model of the methods named flatMap and map on HList, and am open to suggestions for something better.
Now we need a type class that will allow us to take a Poly2, partially apply it to something, and map the resulting unary function over an HList:
trait ApplyMapper[HF, A, X <: HList, Out <: HList] {
def apply(a: A, x: X): Out
}
object ApplyMapper {
implicit def hnil[HF, A] = new ApplyMapper[HF, A, HNil, HNil] {
def apply(a: A, x: HNil) = HNil
}
implicit def hlist[HF, A, XH, XT <: HList, OutH, OutT <: HList](implicit
pb: Poly.Pullback2Aux[HF, A, XH, OutH],
am: ApplyMapper[HF, A, XT, OutT]
) = new ApplyMapper[HF, A, XH :: XT, OutH :: OutT] {
def apply(a: A, x: XH :: XT) = pb(a, x.head) :: am(a, x.tail)
}
}
And now a type class to help with the lifting:
trait LiftA2[HF, X <: HList, Y <: HList, Out <: HList] {
def apply(x: X, y: Y): Out
}
object LiftA2 {
implicit def hnil[HF, Y <: HList] = new LiftA2[HF, HNil, Y, HNil] {
def apply(x: HNil, y: Y) = HNil
}
implicit def hlist[
HF, XH, XT <: HList, Y <: HList,
Out1 <: HList, Out2 <: HList, Out <: HList
](implicit
am: ApplyMapper[HF, XH, Y, Out1],
lift: LiftA2[HF, XT, Y, Out2],
prepend : PrependAux[Out1, Out2, Out]
) = new LiftA2[HF, XH :: XT, Y, Out] {
def apply(x: XH :: XT, y: Y) = prepend(am(x.head, y), lift(x.tail, y))
}
}
And finally our method itself:
def liftA2[HF, X <: HList, Y <: HList, Out <: HList](hf: HF)(x: X, y: Y)(implicit
lift: LiftA2[HF, X, Y, Out]
) = lift(x, y)
And that's all—now liftA2(tuple)(xs, ys) works.
scala> type Result =
| (Int, Double) :: (Int, String) ::
| (Symbol, Double) :: (Symbol, String) ::
| (Char, Double) :: (Char, String) :: HNil
defined type alias Result
scala> val res: Result = liftA2(tuple)(xs, ys)
res: Result = (1,4.0) :: (1,e) :: ('b,4.0) :: ('b,e) :: (c,4.0) :: (c,e) :: HNil
Just as we wanted.

How to add unzipWithIndex to all Scala collections where it makes sense

Suppose I have a list of strings and I use zipWithIndex to transform it to a list of tuples:
List("a", "b", "c").zipWithIndex
res1: List[(java.lang.String, Int)] = List((a,0), (b,1), (c,2))
I'd like to write an unzipWithIndex method that performs the reverse transformation, i.e. a method which, when applied to a list of tuples whose second elements are a permutation of the first N integers, returns the first elements in unpermuted order:
List(("c",2), ("a",0), ("b",1)).unzipWithIndex
res2: List[java.lang.String] = List(a, b, c)
The method should work on any suitable collection of 2-tuples whose second elements are of type Int, preferably using the Pimp My Library pattern. How would I go about this with Scala 2.8 collections?
object Test {
import collection.generic.CanBuildFrom
class Unzip[T, CC[X] <: Traversable[X]]
(coll: CC[(T, Int)])
(implicit bf: CanBuildFrom[CC[(T, Int)], T, CC[T]]) {
def unzipWithIndex: CC[T] = bf(coll) ++= (coll.toSeq sortBy (_._2) map (_._1)) result
}
implicit def installUnzip[T, CC[X] <: Traversable[X]]
(coll: CC[(T, Int)])
(implicit bf: CanBuildFrom[CC[(T, Int)], T, CC[T]]) = new Unzip[T, CC](coll)
def main(args: Array[String]): Unit = {
val x1 = util.Random.shuffle("abcdefgh".zipWithIndex)
println("x1 shuffled = " + x1)
println("x1.unzipWithIndex = " + x1.unzipWithIndex)
val x2 = (1 to 10).toSet.zipWithIndex
println("x2 = " + x2)
println("x2.unzipWithIndex = " + x2.unzipWithIndex)
}
}
% scala Test
x1 shuffled = Vector((f,5), (g,6), (c,2), (d,3), (e,4), (a,0), (h,7), (b,1))
x1.unzipWithIndex = Vector(a, b, c, d, e, f, g, h)
x2 = Set((8,8), (2,5), (3,7), (5,0), (9,4), (4,9), (6,3), (10,1), (7,6), (1,2))
x2.unzipWithIndex = Set(5, 10, 1, 6, 9, 2, 7, 3, 8, 4)
Not an exact match, but you could draw some ideas from the unzip implementation of HList (an arbitrary-length tuple2, coming from Haskell Strongly typed heterogeneous collections).
See Type-Level Programming in Scala, Part 6d: HList Zip/Unzip from Rúnar Óli and Mark Harrah .
we’ll define an unzip type class that accepts an HList of tuples and separates it into two HLists by components:
trait Unzip[H <: HList, R1 <: HList, R2 <: HList] {
def unzip(h: H): (R1, R2)
}
Unzipping HNil produces HNil.
implicit def unzipNil =
new Unzip[HNil, HNil, HNil] {
def unzip(h: HNil) = (HNil, HNil)
}
For HCons, we unzip the tail, separate the head components, and prepend the respective head component to each tail component.
implicit def unzipCons[H1, H2, T <: HList, TR1 <: HList, TR2 <: HList]
(implicit unzipTail: Unzip[T, TR1, TR2]) =
new Unzip[(H1,H2) :: T, H1 :: TR1, H2 :: TR2] {
def unzip(h: (H1,H2) :: T) = {
val (t1, t2) = unzipTail.unzip(h.tail)
(HCons(h.head._1, t1), HCons(h.head._2, t2))
}
}
def unzip[H <: HList, R1 <: HList, R2 <: HList](h: H)(implicit un: Unzip[H, R1, R2]): (R1, R2) =
un unzip h
}
Again, we just need to hook this into our HListOps type class.
Building on the example from above,
// unzip the zipped HLists
val (cc1, cc2) = cc.unzip
val (ca, cb) = cc1.unzip
Here is my take:
implicit def toUnzippable[E](l:List[(E,Int)]) = new AnyRef {
def unzipWithIndex:List[E] = l.sortBy(_._2).map(_._1)
}
This does what your example requires. However, there are two problems with it:
Does not handle missing indices (e.g. List(("a",0), ("b",2)).unzipWithIndex == List("a","b"))
Only works with lists.
You can actually make it to work with all sequences that have builders, so they return the same type of sequence which you put in. But that would require some more work on it.