Convert Sized[Something[String], _] collection to HList - scala

I'm new to shapeless and trying to learning it by doing. I want to make a very small library which can convert a collection (in first step a Sized collection) of String, to HList of different types.
basically what I want to achieve:
import shapeless._
import nat._
import BigQueryParser._
val s: Sized[IndexedSeq[String], nat._3] = Sized("Testing", "2.0", "1")
BigQueryParser[Sized[IndexedSeq[String], nat._3], String :: BigDecimal :: BigInt :: HNil].parse(s)
My non-working implementation is here https://gist.github.com/taojang/f6a9352dbc618039e3a3
I implemented it after https://github.com/milessabin/shapeless/blob/master/examples/src/main/scala/shapeless/examples/csv.scala
My code does not compile, compiler complains about following errors:
[error] /somepath/some-file.scala: could not find implicit value for parameter st: exmaple.BigQueryParser[shapeless.Sized[IndexedSeq[String],shapeless.nat._3],shapeless.::[String,shapeless.::[BigDecimal,shapeless.::[BigInt,shapeless.HNil]]]]
[error] BigQueryParser[Sized[IndexedSeq[String], nat._3], String :: BigDecimal :: BigInt :: HNil].parse(s)

I find that the best way to debug these kinds of issues is to try to build your instances by hand. For example, this is fine:
scala> BigQueryParser[Sized[IndexedSeq[String], _0], HNil]
res0: BigQueryParser[shapeless.Sized[IndexedSeq[String],shapeless.nat._0],shapeless.HNil] = BigQueryParser$$anon$6#2f4cd46d
But this breaks:
scala> deriveHCons[IndexedSeq[String], _0, String, HNil]
<console>:31: error: could not find implicit value for parameter conv: BigQueryParser[scala.collection.generic.IsTraversableLike[IndexedSeq[String]]#A,String]
deriveHCons[IndexedSeq[String], _0, String, HNil]
^
Which suggests that something is going wrong with the IsTraversableLike[Repr]#A type projection. The first thing I'd do in a situation like that is make it a type parameter (ReprA in this case) and then constrain the IsTraversableLike[Repr] instance with a refinement type:
implicit def deriveHCons[Repr, ReprA, L <: Nat, V, T <: HList](implicit
itl: IsTraversableLike[Repr] { type A = ReprA },
ev: AdditiveCollection[Repr],
ts: BigQueryParser[Sized[Repr, L], T],
conv: BigQueryParser[ReprA, V]
): BigQueryParser[Sized[Repr, Succ[L]], V :: T] =
new BigQueryParser[Sized[Repr, Succ[L]], V :: T] {
def parse(s: Sized[Repr, Succ[L]]): Try[V :: T] =
for {
h <- conv.parse(s.head)
t <- ts.parse(s.tail)
} yield h :: t
}
And that works just fine:
scala> println(
| BigQueryParser[
| Sized[IndexedSeq[String], nat._3],
| String :: BigDecimal :: BigInt :: HNil
| ].parse(s)
| )
Success(Testing :: 2.0 :: 1 :: HNil)
There may be other simplifications you could make (e.g. using _0 directly instead of a L <: _0 type parameter), but this fix should at least get things rolling.

Related

Is there a way to use the type of an object as the argument of a type parameter?

I'm trying to use shapeless to do additions and removals from Hlists. But I can't seem to get it to work.
So I here are my lists:
object ShapelessExample {
import shapeless._
def main(args: Array[String]): Unit = {
case class Container[T <: Singleton](name: T)
val a = Container("A") :: Container("B") :: Container("C") :: HNil
val b = Container("B") :: Container("C") :: HNil
println {
a.removeAll[b.type] //doesn't work
}
}
}
So the removeAll method on Hlist only takes a type parameter, but I can't seem to use b.type. I can manually specify a.removeAll[Container["B"] :: Container["C"] :: HNil], but is there any way to just use b's type?
Shapeless tries to remove precisely type b.type but can't find it among Container["A"] :: Container["B"] :: Container["C"] :: HNil so #user is correct, singleton type b.type is too specific.
In order to infer an HList type from a val singleton type try to modify the method
implicit class RemoveAllOps[L <: HList](a: L) {
def removeAll[L1 <: HList](b: L1)(implicit
ra: shapeless.ops.hlist.RemoveAll[L, L1]
): ra.Out = ra(a)
}
a.removeAll(b) // (Container(B) :: Container(C) :: HNil,Container(A) :: HNil)
or
implicit class RemoveAllOps[L <: HList](a: L) {
def removeAllFrom[O <: Singleton] = new {
def apply[L1 >: O <: HList]()(implicit
ra: shapeless.ops.hlist.RemoveAll[L, L1]
): ra.Out = ra(a)
}
}
a.removeAllFrom[b.type]() //(Container(B) :: Container(C) :: HNil,Container(A) :: HNil)

Taking HList of Seq[_] and generating Seq[HList] with cartesian product of values

I'm new to Shapless.
I'm trying to write a function that would take an HList of sequences of different types, convert it to a Seq[HList] containing the cartesian product of the original HList's elements, and iterate over the resulting Sequence
For example:
val input = Seq(true, false) :: Seq(1,2,3)::Seq("foo", "bar") :: HNil
cartesianProduct: Seq[Boolean :: Int :: String :: HNil] = Seq(
true :: 1 :: foo :: HNil,
true :: 1 :: bar :: HNil,
true :: 2 :: foo :: HNil,
true :: 2 :: bar :: HNil,
true :: 3 :: foo :: HNil,
true :: 3 :: bar :: HNil,
false :: 1 :: foo :: HNil,
false :: 1 :: bar :: HNil,
false :: 2 :: foo :: HNil,
false :: 2 :: bar :: HNil,
false :: 3 :: foo :: HNil,
false :: 3 :: bar :: HNil)
I was able to achieve this in Intellij Scala Worksheet using the following code:
import shapeless._
import shapeless.ops.hlist.LeftFolder
object combine extends Poly {
implicit def `case`[T <: HList, S] = use((acc : Seq[T], curr : Seq[S]) => {
for {
el <- curr
v <- acc
} yield el :: v
})
}
val input = Seq(true, false) :: Seq(1,2,3)::Seq("foo", "bar") :: HNil
val combinations = input.foldLeft(Seq[HNil](HNil))(combine)
combinations.foreach(println)
Here everything works, I assume, because the full type of input is known to the compiler.
However, when I try to wrap the whole operation in a function, the full type of input gets lost, and I can't call foreach on the result of foldLeft:
def cartesian[T <: HList](input: T)
(implicit folder: LeftFolder[T, Seq[HNil], combine.type]) = {
input.foldLeft(Seq[HNil](HNil))(combine)
.foreach(println)
}
The compiler complains:
value foreach is not a member of folder.Out
input.foldLeft(Seq[HNil](HNil))(combine).foreach(println)
^
I imagine there is some implicit evidence I can request to assert the correct shape of input (HList of Seq[_]) and thus get the compiler to figure out the resulting type of the foldLeft, but I can't figure out what it could be...
Hope someone can help me figure this out.
Thanks.
Update:
My eventual goal with this question was, given an HList of Seq[_] to derive a function (perhaps on a case class) that will accept a function with the same arg arity as the input HList and the argument types matching the 'Seq' element types in the same order. For example for the input above the function would be f: (Boolean, Int, String) => R
So in affect I would be able to iterate over the cartesian product of the input using f.
The final code looks like this:
import shapeless._
import shapeless.ops.function.FnToProduct
import shapeless.ops.hlist.LeftFolder
import shapeless.syntax.std.function.fnHListOps
object combine extends Poly {
implicit def `case`[EL <: HList, S] = use((acc : Seq[EL], curr : Seq[S]) => {
for {
el <- curr
v <- acc
} yield el :: v
})
}
case class Cartesian[R <: HList, F, FR](combinations: Seq[R])
(implicit ftp: FnToProduct.Aux[F, R => Unit]) {
def foreach(f: F) = combinations.foreach(f.toProduct)
}
def cartesian[T <: HList, R <: HList, F, FR](variants: T)(implicit
folder: LeftFolder.Aux[T, Seq[HNil], combine.type, _ <: Seq[R]],
fnToProd: FnToProduct.Aux[F, R => Unit]
) = {
val combinations: Seq[R] = variants.foldLeft(Seq[HNil](HNil))(combine)
Cartesian(combinations)
}
val variants = Seq(true, false) :: Seq("foo", "bar") :: Seq(1, 2, 3) :: HNil
cartesian(variants).foreach((a, b, c) => println(s"$a, $b, $c"))
Note that the types for the function arguments a, b, c are correctly inferred and are Boolean, String, and Int.
Currently the result type of the function passed into foreach has to be fixed (in the code above it is Unit). It can't be inferred from the function passed in.
The problem is not that compiler doesn't know anything about input, but rather that it does not know anything about output.
Inside def cartesian all that is known to compiler is that after foldLeft you get some type folder.Out (which depends on an instance that shapeless would figure for you).
For ensuring result types, you can use LeftFolder.Aux with one extra type parameter, e.g.
def cartesian[T <: HList](input: T)
(implicit folder: LeftFolder.Aux[T, Seq[HNil], combine.type, _ <: Seq[Any]]) = {
input.foldLeft(Seq[HNil](HNil))(combine)
.foreach(println)
}
Now the compiler will know that result is some subtype of Seq[Any], so it is possible to call foreach on it.
Of course, this is only the problem inside the def. At call sites output types will be resolved based on input, so you would be able to do this without Aux:
def cartesian2[T <: HList](input: T)
(implicit folder: LeftFolder[T, Seq[HNil], combine.type]) = {
input.foldLeft(Seq[HNil](HNil))(combine)
}
cartesian2(input).foreach(println)
Runnable code: https://scalafiddle.io/sf/n409yNW/2

List of HLists to HList of Lists?

Basically I want to transpose my object. Is there a simple way to do this? If my HList is large and I don't want to fold.
This is in service of unzipping a List of a high-arity tuples.
I don't know of an existing solution, but it seemed to be an interesting task, so I tried my hand at it.
Assuming we have a list of HLists, like
val myList: List[Int :: String :: Boolean :: HNil] = List(
1 :: "sample1" :: true :: HNil,
2 :: "sample2" :: false :: HNil,
3 :: "sample3" :: true :: HNil
)
and we want a HList of Lists, like
val expected: List[Int] :: List[String] :: List[Boolean] :: HNil =
List(3, 2, 1) ::
List("sample3", "sample2", "sample") ::
List(true, false, true) ::
HNil
we need to create one typeclass. I named it EmptyOf. It has a single method that creates an HList containing empty lists.
trait EmptyOf[T <: HList, AsList <: HList] {
def empty: AsList
}
Here is the implementation for HNil:
implicit val emptyOfHNil: EmptyOf[HNil, HNil] = () => HNil
And now for HCons:
implicit def emptyOfHCons[H, T <: HList, TEmpty <: HList](
implicit ev: EmptyOf[T, TEmpty]
): EmptyOf[H :: T, List[H] :: TEmpty] = () => List.empty[H] :: ev.empty()
So, for EmptyOf[T :: U :: HNil, List[T] :: List[U] :: HNil] our instance's empty will return Nil :: Nil :: HNil.
With these, we can implement transpose by folding on the list of HLists and using a Poly2 to merge the HLists.
object concat extends Poly2 {
implicit def prepend[S] = at[List[S], S]((list, s) => s :: list)
}
def transpose[T <: HList, U <: HList](lists: List[T])
(implicit emptyOf: EmptyOf[T, U],
zip: ZipWith.Aux[U, T, concat.type, U]): U =
lists.foldLeft(emptyOf.empty())(_.zipWith(_)(concat))
Now transpose(myList) should be equal to expected (and of the correct type).

Scala collection of classes with Implicit Ordering

I would like to create an Array (or List, ArrayBuffer, etc) which can only contain instance of classes with defined implicit Ordering (e.g. Int, Long, Double).
Something like this:
val ab = new ArrayBuffer[???]()
ab += 7
ab += 8.9
ab += 8L
I don't want to compare these values with each other.
Just use the type class constraint as shown below
def createList[T: Ordering](values: T*) = values.toList
T: Ordering implies only type which has Ordering instance in scope are allowed to be passed as arguments to the function.
scala> def createList[T: Ordering](values: T*) = values.toList
createList: [T](values: T*)(implicit evidence$1: Ordering[T])List[T]
scala> case class Cat()
defined class Cat
scala> createList(1, 2, 3)
res2: List[Int] = List(1, 2, 3)
scala> createList(Cat())
<console>:15: error: No implicit Ordering defined for Cat.
createList(Cat())
^
Integer ordering is available in scope but cat ordering is not available in scope. So you cannot pass Cat values until you provide instance of Ordering[Cat]
Now lets provide some fake ordering and see if compiler accepts Cat as argument
scala> implicit val orderingCat: Ordering[Cat] = (a: Cat, b: Cat) => ???
orderingCat: Ordering[Cat] = $anonfun$1#6be766d1
scala> createList(Cat())
res4: List[Cat] = List(Cat())
It works.
If you really want to have a list of objects of different types and still be able to statically check things about the objects in that list at compile time, you would have to use something like a HList from shapeless. Here is an example of how you could have two heterogeneous lists and still check at compile time that each ith element of both lists can be compared with each other.
import shapeless._
import shapeless.ops.hlist.{LiftAll, Zip, Mapper}
object lt extends Poly1 {
implicit def instance[A] = at[(Ordering[A], A, A)] {
case (ord, a, b) => ord.lt(a, b)
}
}
def areLessThan[L <: HList, O <: HList, OLL <: HList](a: L, b: L)(
implicit
ord: LiftAll.Aux[Ordering, L, O],
zip: Zip.Aux[O :: L :: L :: HNil, OLL],
map: Mapper[lt.type, OLL]
) = zip(ord.instances :: a :: b :: HNil).map(lt)
Using it:
scala> val a = 1 :: "b" :: Option(4L) :: HNil
a: Int :: String :: Option[Long] :: shapeless.HNil = 1 :: b :: Some(4) :: HNil
scala> val b = 2 :: "a" :: Option(7L) :: HNil
b: Int :: String :: Option[Long] :: shapeless.HNil = 2 :: a :: Some(7) :: HNil
scala> areLessThan(a, b)
res10: Boolean :: Boolean :: Boolean :: shapeless.HNil = true :: false :: true :: HNil

Weird behavior trying to convert case classes to heterogeneous lists recursively with Shapeless

I stayed up way too late last night trying to figure out this Shapeless issue and I'm afraid it's going to eat my evening if I don't get it off my chest, so here goes.
In this minimized version I'm just defining a type class that will recursively convert case classes into heterogeneous lists:
import shapeless._
trait DeepHLister[R <: HList] extends DepFn1[R] { type Out <: HList }
trait LowPriorityDeepHLister {
type Aux[R <: HList, Out0 <: HList] = DeepHLister[R] { type Out = Out0 }
implicit def headNotCaseClassDeepHLister[H, T <: HList](implicit
dht: DeepHLister[T]
): Aux[H :: T, H :: dht.Out] = new DeepHLister[H :: T] {
type Out = H :: dht.Out
def apply(r: H :: T) = r.head :: dht(r.tail)
}
}
object DeepHLister extends LowPriorityDeepHLister {
implicit object hnilDeepHLister extends DeepHLister[HNil] {
type Out = HNil
def apply(r: HNil) = HNil
}
implicit def headCaseClassDeepHLister[H, R <: HList, T <: HList](implicit
gen: Generic.Aux[H, R],
dhh: DeepHLister[R],
dht: DeepHLister[T]
): Aux[H :: T, dhh.Out :: dht.Out] = new DeepHLister[H :: T] {
type Out = dhh.Out :: dht.Out
def apply(r: H :: T) = dhh(gen.to(r.head)) :: dht(r.tail)
}
def apply[R <: HList](implicit dh: DeepHLister[R]): Aux[R, dh.Out] = dh
}
Let's try it out! First we need some case classes:
case class A(x: Int, y: String)
case class B(x: A, y: A)
case class C(b: B, a: A)
case class D(a: A, b: B)
And then (note that I've cleaned up the type syntax for the sake of this not being a totally unreadable mess):
scala> DeepHLister[A :: HNil]
res0: DeepHLister[A :: HNil]{
type Out = (Int :: String :: HNil) :: HNil
} = DeepHLister$$anon$2#634bf0bf
scala> DeepHLister[B :: HNil]
res1: DeepHLister[B :: HNil] {
type Out = (
(Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil
) :: HNil
} = DeepHLister$$anon$2#69d6b3e1
scala> DeepHLister[C :: HNil]
res2: DeepHLister[C :: HNil] {
type Out = (
((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) ::
(Int :: String :: HNil) ::
HNil
) :: HNil
} = DeepHLister$$anon$2#4d062faa
So far so good. But then:
scala> DeepHLister[D :: HNil]
res3: DeepHLister[D :: HNil] {
type Out = ((Int :: String :: HNil) :: B :: HNil) :: HNil
} = DeepHLister$$anon$2#5b2ab49a
The B didn't get converted. If we turn on -Xlog-implicits this is the last message:
<console>:25: this.DeepHLister.headCaseClassDeepHLister is not a valid implicit value for DeepHLister[shapeless.::[B,shapeless.HNil]] because:
hasMatchingSymbol reported error: diverging implicit expansion for type DeepHLister[this.Repr]
starting with method headNotCaseClassDeepHLister in trait LowPriorityDeepHLister
DeepHLister[D :: HNil]
^
Which doesn't make sense to me—headCaseClassDeepHLister should be able to generate DeepHLister[B :: HNil] just fine, and it does if you ask it directly.
This happens on both 2.10.4 and 2.11.2, and with both the 2.0.0 release and master. I'm pretty sure this has to be a bug, but I'm not ruling out the possibility that I'm doing something wrong. Has anyone seen anything like this before? Is there something wrong with my logic or some restriction on Generic I'm missing?
Okay, thanks for listening—maybe now I can go read a book or something.
This now works more or less as written using recent shapeless-2.1.0-SNAPSHOT builds, and a close relative of the sample in this question has been added there as an example.
The problem with the original is that each expansion of a Generic introduces a new HList type into the implicit resolution of the DeepHLister type class instances and, in principle, could produce an HList type that is related to but more complex than some type seen previously during the same resolution. This condition trips the divergence checker and aborts the resolution process.
The exact details of why this happens for D but not for C is lurking in the details of the implementation of Scala's typechecker but, to a rough approximation, the differentiator is that during the resolution for C we see the B (larger) before the A (smaller) so the divergence checker is happy that our types are converging; conversely during the resolution for D we see the A (smaller) before the B (larger) so the divergence checker (conservatively) bails.
The fix for this in shapeless 2.1.0 is the recently enhanced Lazy type constructor and associated implicit macro infrastructure. This allows much more user control over divergence and supports the use of implicit resolution to construct the recursive implicit values which are crucial to the ability to automatically derive type class instances for recursive types. Many examples of this can be found in the shapeless code base, in particular the reworked type class derivation infrastructure and Scrap Your Boilerplate implementation, which no longer require dedicated macro support, but are implemented entirely in terms of the Generic and Lazy primitives. Various applications of these mechanisms can be found in the shapeless examples sub-project.
I took a slightly different approach.
trait CaseClassToHList[X] {
type Out <: HList
}
trait LowerPriorityCaseClassToHList {
implicit def caseClass[X](implicit gen: Generic[X]): CaseClassToHList[X] {
type Out = generic.Repr
} = null
}
object CaseClassToHList extends LowerPriorityCaseClassToHList {
type Aux[X, R <: HList] = CaseClassToHList[X] { type Out = R }
implicit def caseClassWithCaseClasses[X, R <: HList](
implicit toHList: CaseClassToHList.Aux[X, R],
nested: DeepHLister[R]): CaseClassToHList[X] {
type Out = nested.Out
} = null
}
trait DeepHLister[R <: HList] {
type Out <: HList
}
object DeepHLister {
implicit def hnil: DeepHLister[HNil] { type Out = HNil } = null
implicit def caseClassAtHead[H, T <: HList](
implicit head: CaseClassToHList[H],
tail: DeepHLister[T]): DeepHLister[H :: T] {
type Out = head.Out :: tail.Out
} = null
def apply[X <: HList](implicit d: DeepHLister[X]): d.type = null
}
Tested with the following code:
case class A(x: Int, y: String)
case class B(x: A, y: A)
case class C(b: B, a: A)
case class D(a: A, b: B)
object Test {
val z = DeepHLister[HNil]
val typedZ: DeepHLister[HNil] {
type Out = HNil
} = z
val a = DeepHLister[A :: HNil]
val typedA: DeepHLister[A :: HNil] {
type Out = (Int :: String :: HNil) :: HNil
} = a
val b = DeepHLister[B :: HNil]
val typedB: DeepHLister[B :: HNil] {
type Out = ((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil
} = b
val c = DeepHLister[C :: HNil]
val typedC: DeepHLister[C :: HNil] {
type Out = (((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil
} = c
val d = DeepHLister[D :: HNil]
val typedD: DeepHLister[D :: HNil] {
type Out = ((Int :: String :: HNil) :: ((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil) :: HNil
} = d
}