Shapeless HList polymorphic map with an argument - scala

Given an HList of Label[A](String) I want to map it into an HList of LabelWithValue[A](Label[A], A), where the actual values come from a Map[String, Any]. In the example below I just defined the map of values in the method, just imagine the values come from a database.
The below works, but it is very veeery hacky because it uses a global var. Instead I'd like to pass the Map[String, Any] into GetLabelWithValue. I didn't find a way though, because the caller of getValues implicitly creates a Mapper, and at that point the map of values doesn't exist yet. I tried to create a Mapper myself, but my type level programming skills aren't yet good enough.
import shapeless._
import shapeless.poly._
import shapeless.ops.hlist._
object Main extends App {
case class Label[A](name: String)
case class LabelWithValue[A](label: Label[A], value: A)
// TODO: avoid the horrible global state - pass in the Map as a parameter
var horribleGlobalState: Map[String, Any] = _
object GetLabelWithValue extends (Label ~> LabelWithValue) {
def apply[A](label: Label[A]) =
LabelWithValue(label, horribleGlobalState.get(label.name).asInstanceOf[A])
}
val label1 = Label[Int]("a")
val label2 = Label[String]("b")
val labels = label1 :: label2 :: HNil
val labelsWithValues: LabelWithValue[Int] :: LabelWithValue[String] :: HNil = getValues(labels)
println(labelsWithValues)
def getValues[L <: HList, M <: HList](labels: L)(
implicit mapper: Mapper.Aux[GetLabelWithValue.type, L, M]) = {
horribleGlobalState = Map("a" -> 5, "b" -> "five")
labels map GetLabelWithValue
}
}
Here is an alternative implementation of GetLabelWithValue, which behaves the same way:
object GetLabelWithValue extends Poly1 {
implicit def caseLabel[A] = at[Label[A]] { label ⇒
LabelWithValue(label, horribleGlobalState.get(label.name).asInstanceOf[A])
}
}

I am by no means shapeless guru but here's first thing that comes to my mind:
object Main extends App {
case class Label[A](name: String)
case class LabelWithValue[A](label: Label[A], value: A)
object combine extends Poly2 {
implicit def workS[A <: HList, B] = at[Label[B], (Map[String, Any], A)] {
case (i, (map, res)) ⇒
(map, LabelWithValue(i, map.get(i.name).asInstanceOf[B]) :: res)
}
}
var state: Map[String, Any] = Map("a" -> 5, "b" -> "five")
val label1 = Label[Int]("a")
val label2 = Label[String]("b")
val labels = label1 :: label2 :: HNil
val mapped = labels.foldRight((state, HNil))(combine)._2
println(mapped)
}
I'm not saying there's not better way, but this seems pretty reasonable - instead of global state you capture it using fold and decide based on it. Probably gives you a bit more power than you need though (as you could mutate the map inbetween folds, but...)

Here's the full solution (based on KadekM's solution) when you want to use it in a method. The hard bit was to extract the type out of the tuple (which is the result of the fold).
import shapeless._
import shapeless.ops.hlist._
import shapeless.ops.tuple.IsComposite
object Main extends App {
case class Label[A](name: String)
case class LabelWithValue[A](label: Label[A], value: A)
object combineLabelWithValue extends Poly2 {
implicit def atLabel[A, B <: HList] = at[Label[A], (B, Map[String, Any])] {
case (label, (acc, values)) ⇒
(LabelWithValue(label, values.get(label.name).asInstanceOf[A]) :: acc, values)
}
}
val label1 = Label[Int]("a")
val label2 = Label[String]("b")
val labels = label1 :: label2 :: HNil
val labelsWithValues: LabelWithValue[Int] :: LabelWithValue[String] :: HNil = getValues(labels)
println(labelsWithValues)
def getValues[L <: HList, Out, P](labels: L)(
implicit folder: RightFolder.Aux[L, (HNil.type, Map[String, Any]), combineLabelWithValue.type, P],
ic: IsComposite.Aux[P, Out, _]
): Out = {
val state = Map("a" -> 5, "b" -> "five")
val resultTuple = labels.foldRight((HNil, state))(combineLabelWithValue)
ic.head(resultTuple)
}
}

Related

Getting elements from Slick HLIST (or Convert Slick HLIst into Shapeless HList)

I have auto-generated scala code using slick codegen. I see that some tables Rows are implements as HLists. (but these are slick HList, not the normal shapeless HList)
Now I want a specific element from the HList returned as a Row by the slick query.
I googled and found this thread
Getting elements from an HList
But this does not work for slick HList. it works very well for Shapeless HList
I also tried the apply method
val x : Long = slickHList(2)
but this doesn't compile because type Any does not conform to exected type of Long. I would hate to do a .asInstanceOf
Is there a typesafe way in which I can access the elements of the slick HList?
Edit: Based on the input below I wrote the code below
package com.abhi
object SlickAndShapeless {
import slick.collection.heterogeneous.{HCons, HList, HNil}
import slick.collection.heterogeneous.syntax.HNil
type MyRow = HCons[Long, HCons[String, HNil]]
val row : MyRow = 1L :: "foo" :: HNil
import HListExtensions._
val hlist = row.asShapeless
val container = new Container(hlist)
val item = container.get(1)
}
class Container[L <: shapeless.HList](list: L) {
import shapeless._
import nat._
import ops.hlist._
def get(n: Nat)(implicit at: At[L, n.N]): at.Out = list[n.N]
}
object HListExtensions {
import slick.collection.heterogeneous.{HNil => SHNil, HList => SHList, HCons}
import shapeless.{::, HList, HNil}
implicit class HListShapelessSlick(val list: HList) extends AnyVal {
def asSlick : SHList = list match {
case HNil => SHNil
case head :: tail => head :: tail.asSlick
}
}
implicit class HListSlickShapeless(val list: SHList) extends AnyVal {
def asShapeless : HList = list match {
case SHNil => HNil
case HCons(head, tail) => head :: tail.asShapeless
}
}
}
The problem with the code above is that the type of item obtained from val item = container.get(1) is at.Out and not Long as I was expecting.
build.sbt
libraryDependencies ++= Seq(
"com.typesafe.slick" % "slick_2.12" % "3.2.1",
"com.chuusai" % "shapeless_2.12" % "2.3.2"
)
I also see two compiler errors
Error:(19, 35) Implicit not found: shapeless.Ops.At[shapeless.HList, shapeless.Succ[shapeless._0]]. You requested to access an element at the position shapeless.Succ[shapeless._0], but the HList shapeless.HList is too short.
val item : Long = container.get(1)
Error:(19, 35) not enough arguments for method get: (implicit at: shapeless.ops.hlist.At[shapeless.HList,shapeless.Succ[shapeless._0]])at.Out.
Unspecified value parameter at.
val item : Long = container.get(1)
It's possible to create extension methods:
object HListExtensions {
import slick.collection.heterogeneous.{HNil => SHNil, HList => SHList, HCons}
import shapeless.{ ::, HList, HNil }
implicit class HListShapelessSlick(val list:HList) extends AnyVal {
def asSlick:SHList = list match {
case HNil => SHNil
case head :: tail => head :: tail.asSlick
}
}
implicit class HListSlickShapeless(val list:SHList) extends AnyVal {
def asShapeless:HList = list match {
case SHNil => HNil
case HCons(head, tail) => head :: tail.asShapeless
}
}
}
Example:
scala>import HListExtensions._
import HListExtensions._
scala> val x1:HList = 1 :: 2 :: HNil
x1: slick.collection.heterogeneous.HList = 1 :: 2 :: HNil
scala> x1.asShapeless
res1: shapeless.HList = 1 :: 2 :: HNil
scala> x1.asShapeless.asSlick
res2: slick.collection.heterogeneous.HList = 1 :: 2 :: HNil
I hope this helps.
Edit: Here is type level solution.
object HListsConvertersTypeLevel {
import shapeless.{::}
sealed trait HConv[From <: heterogeneous.HList, To <: shapeless.HList] {
def convert(list: From): To
}
implicit def buildHConvNil: HConv[heterogeneous.HNil.type, shapeless.HNil] =
new HConv[heterogeneous.HNil.type, shapeless.HNil] {
override def convert(list: heterogeneous.HNil.type): shapeless.HNil = shapeless.HNil
}
implicit def buildHConv[H, T <: heterogeneous.HList, T2 <: shapeless.HList](
implicit conv: HConv[T, T2]): HConv[HCons[H, T], ::[H, T2]] = new HConv[HCons[H, T], ::[H, T2]] {
override def convert(list: HCons[H, T]): ::[H, T2] = {
list.head :: conv.convert(list.tail)
}
}
def toShapeless[A <: heterogeneous.HList, B <: shapeless.HList](list: A)(implicit conv: HConv[A, B]): B = conv.convert(list)
}
Example:
object SlickAndShapeless {
import slick.collection.heterogeneous.{HCons, HNil}
import slick.collection.heterogeneous.syntax.HNil
type MyRow = HCons[Long, HCons[String, HNil]]
val row: MyRow = 1L :: "foo" :: HNil
import HListsConvertersTypeLevel._
val hlist = toShapeless(row)
val item: Long = hlist.head
val item2: String = hlist.tail.head
}

Scala: reflection and case classes

The following code succeeds, but is there a better way of doing the same thing? Perhaps something specific to case classes? In the following code, for each field of type String in my simple case class, the code goes through my list of instances of that case class and finds the length of the longest string of that field.
case class CrmContractorRow(
id: Long,
bankCharges: String,
overTime: String,
name$id: Long,
mgmtFee: String,
contractDetails$id: Long,
email: String,
copyOfVisa: String)
object Go {
def main(args: Array[String]) {
val a = CrmContractorRow(1,"1","1",4444,"1",1,"1","1")
val b = CrmContractorRow(22,"22","22",22,"55555",22,"nine long","22")
val c = CrmContractorRow(333,"333","333",333,"333",333,"333","333")
val rows = List(a,b,c)
c.getClass.getDeclaredFields.filter(p => p.getType == classOf[String]).foreach{f =>
f.setAccessible(true)
println(f.getName + ": " + rows.map(row => f.get(row).asInstanceOf[String]).maxBy(_.length))
}
}
}
Result:
bankCharges: 3
overTime: 3
mgmtFee: 5
email: 9
copyOfVisa: 3
If you want to do this kind of thing with Shapeless, I'd strongly suggest defining a custom type class that handles the complicated part and allows you to keep that stuff separate from the rest of your logic.
In this case it sounds like the tricky part of what you're specifically trying to do is getting the mapping from field names to string lengths for all of the String members of a case class. Here's a type class that does that:
import shapeless._, shapeless.labelled.FieldType
trait StringFieldLengths[A] { def apply(a: A): Map[String, Int] }
object StringFieldLengths extends LowPriorityStringFieldLengths {
implicit val hnilInstance: StringFieldLengths[HNil] =
new StringFieldLengths[HNil] {
def apply(a: HNil): Map[String, Int] = Map.empty
}
implicit def caseClassInstance[A, R <: HList](implicit
gen: LabelledGeneric.Aux[A, R],
sfl: StringFieldLengths[R]
): StringFieldLengths[A] = new StringFieldLengths[A] {
def apply(a: A): Map[String, Int] = sfl(gen.to(a))
}
implicit def hconsStringInstance[K <: Symbol, T <: HList](implicit
sfl: StringFieldLengths[T],
key: Witness.Aux[K]
): StringFieldLengths[FieldType[K, String] :: T] =
new StringFieldLengths[FieldType[K, String] :: T] {
def apply(a: FieldType[K, String] :: T): Map[String, Int] =
sfl(a.tail).updated(key.value.name, a.head.length)
}
}
sealed class LowPriorityStringFieldLengths {
implicit def hconsInstance[K, V, T <: HList](implicit
sfl: StringFieldLengths[T]
): StringFieldLengths[FieldType[K, V] :: T] =
new StringFieldLengths[FieldType[K, V] :: T] {
def apply(a: FieldType[K, V] :: T): Map[String, Int] = sfl(a.tail)
}
}
This looks complex, but once you start working with Shapeless a bit you learn to write this kind of thing in your sleep.
Now you can write the logic of your operation in a relatively straightforward way:
def maxStringLengths[A: StringFieldLengths](as: List[A]): Map[String, Int] =
as.map(implicitly[StringFieldLengths[A]].apply).foldLeft(
Map.empty[String, Int]
) {
case (x, y) => x.foldLeft(y) {
case (acc, (k, v)) =>
acc.updated(k, acc.get(k).fold(v)(accV => math.max(accV, v)))
}
}
And then (given rows as defined in the question):
scala> maxStringLengths(rows).foreach(println)
(bankCharges,3)
(overTime,3)
(mgmtFee,5)
(email,9)
(copyOfVisa,3)
This will work for absolutely any case class.
If this is a one-off thing, you might as well use runtime reflection, or you could use the Poly1 approach in Giovanni Caporaletti's answer—it's less generic and it mixes up the different parts of the solution in a way I don't prefer, but it should work just fine. If this is something you're doing a lot of, though, I'd suggest the approach I've given here.
If you want to use shapeless to get the string fields of a case class and avoid reflection you can do something like this:
import shapeless._
import labelled._
trait lowerPriorityfilterStrings extends Poly2 {
implicit def default[A] = at[Vector[(String, String)], A] { case (acc, _) => acc }
}
object filterStrings extends lowerPriorityfilterStrings {
implicit def caseString[K <: Symbol](implicit w: Witness.Aux[K]) = at[Vector[(String, String)], FieldType[K, String]] {
case (acc, x) => acc :+ (w.value.name -> x)
}
}
val gen = LabelledGeneric[CrmContractorRow]
val a = CrmContractorRow(1,"1","1",4444,"1",1,"1","1")
val b = CrmContractorRow(22,"22","22",22,"55555",22,"nine long","22")
val c = CrmContractorRow(333,"333","333",333,"333",333,"333","333")
val rows = List(a,b,c)
val result = rows
// get for each element a Vector of (fieldName -> stringField) pairs for the string fields
.map(r => gen.to(r).foldLeft(Vector[(String, String)]())(filterStrings))
// get the maximum for each "column"
.reduceLeft((best, row) => best.zip(row).map {
case (kv1#(_, v1), (_, v2)) if v1.length > v2.length => kv1
case (_, kv2) => kv2
})
result foreach { case (k, v) => println(s"$k: $v") }
You probably want to use Scala reflection:
import scala.reflect.runtime.universe._
val rm = runtimeMirror(getClass.getClassLoader)
val instanceMirrors = rows map rm.reflect
typeOf[CrmContractorRow].members collect {
  case m: MethodSymbol if m.isCaseAccessor && m.returnType =:= typeOf[String] =>
    val maxValue = instanceMirrors map (_.reflectField(m).get.asInstanceOf[String]) maxBy (_.length)
    println(s"${m.name}: $maxValue")
}
So that you can avoid issues with cases like:
case class CrmContractorRow(id: Long, bankCharges: String, overTime: String, name$id: Long, mgmtFee: String, contractDetails$id: Long, email: String, copyOfVisa: String) {
val unwantedVal = "jdjd"
}
Cheers
I have refactored your code to something more reuseable:
import scala.reflect.ClassTag
case class CrmContractorRow(
id: Long,
bankCharges: String,
overTime: String,
name$id: Long,
mgmtFee: String,
contractDetails$id: Long,
email: String,
copyOfVisa: String)
object Go{
def main(args: Array[String]) {
val a = CrmContractorRow(1,"1","1",4444,"1",1,"1","1")
val b = CrmContractorRow(22,"22","22",22,"55555",22,"nine long","22")
val c = CrmContractorRow(333,"333","333",333,"333",333,"333","333")
val rows = List(a,b,c)
val initEmptyColumns = List.fill(a.productArity)(List())
def aggregateColumns[Tin:ClassTag,Tagg](rows: Iterable[Product], aggregate: Iterable[Tin] => Tagg) = {
val columnsWithMatchingType = (0 until rows.head.productArity).filter {
index => rows.head.productElement(index) match {case t: Tin => true; case _ => false}
}
def columnIterable(col: Int) = rows.map(_.productElement(col)).asInstanceOf[Iterable[Tin]]
columnsWithMatchingType.map(index => (index,aggregate(columnIterable(index))))
}
def extractCaseClassFieldNames[T: scala.reflect.ClassTag] = {
scala.reflect.classTag[T].runtimeClass.getDeclaredFields.filter(!_.isSynthetic).map(_.getName)
}
val agg = aggregateColumns[String,String] (rows,_.maxBy(_.length))
val fieldNames = extractCaseClassFieldNames[CrmContractorRow]
agg.map{case (index,value) => fieldNames(index) + ": "+ value}.foreach(println)
}
}
Using shapeless would get rid of the .asInstanceOf, but the essence would be the same. The main problem with the given code was that it was not re-usable since the aggregation logic was mixed with the reflection logic to get the field names.

Mapping tuples in shapeless HList

You do
import shapeless._ ; import poly._
object fun extends (List ~>> (List, Int)) {
override def apply[T](list: List[T]): (List, Int) = list -> list.size
}
println((List(1,2,3) :: List("a", "b", "c") :: HNil).map(fun))
to map every sublist into a couple. However, what if HList elements are more complex, like tutples, for instance? The natural attempt
object fun extends ((String -> List) ~>> (List, Int)) {
override def apply[T](list: (String -> List[T])): (List, Int) = list -> list.size
is rejected by the compiler. What do you do? Where can you learn that?
import shapeless._
import poly._
object fun2 extends Poly1 {
implicit def caseTuple[T] =
at[(String, List[T])](x => (x._2.tail, x._1.toInt))
}
Then:
println((("56", List(1,2,3)) :: ("78", List(4,2,3)) :: HNil).map(fun))
Or if you want to do it with original ~> it's yet still possible, but looks ugly to my taste:
object fun3 extends (({type L[T] = (String, List[T])})#L ~> ({type L[T] = (List[T], Int)})#L) {
override def apply[T](f: (String, List[T])): (List[T], Int) = f match {
case (str, list) => (list.tail, str.toInt)
}
}
println((("56", List(1,2,3)) :: ("78", List(4,2,3)) :: HNil).map(fun3))
P.S. Also please note that your code example does not compile. You miss [T] in the first part, you need to use ~> and String -> List is a wrong type in scala in the last part. List is a type constructor, not type and you can't use it this way

Shapeless HList map after foldRight

A type level foldRight works fine (getLabelWithValues), and a follow-on type level map (getValues) also works fine. If I combine both in one method (getValuesFull), it doesn't work any more though. What is the missing piece?
The full source (with sbt ready to ~run with implicit debug output) is here: https://github.com/mpollmeier/shapeless-playground/tree/8170a5b
case class Label[A](name: String)
case class LabelWithValue[A](label: Label[A], value: A)
val label1 = Label[Int]("a")
val labels = label1 :: HNil
object combineLabelWithValue extends Poly2 {
implicit def atLabel[A, B <: HList] = at[Label[A], (B, Map[String, Any])] {
case (label, (acc, values)) ⇒
(LabelWithValue(label, values(label.name).asInstanceOf[A]) :: acc, values)
}
}
object GetLabelValue extends (LabelWithValue ~> Id) {
def apply[B](labelWithValue: LabelWithValue[B]) = labelWithValue.value
}
val labelsWithValues: LabelWithValue[Int] :: HNil = getLabelWithValues(labels)
// manually mapping it works fine:
val valuesManual: Int :: HNil = labelsWithValues.map(GetLabelValue)
// using a second function with Mapper works fine:
val valuesSecondFn: Int :: HNil = getValues(labelsWithValues)
// error: could not find implicit value for parameter mapper: shapeless.ops.hlist.Mapper.Aux[Main.GetLabelValue.type,WithValues,Values]
// val valuesFull: Int :: HNil = getValuesFull(labels)
def getLabelWithValues[L <: HList, P, WithValues](labels: L)(
implicit folder: RightFolder.Aux[L, (HNil.type, Map[String, Any]), combineLabelWithValue.type, P],
ic: IsComposite.Aux[P, WithValues, _]
): WithValues = {
val state = Map("a" -> 5, "b" -> "five")
val resultTuple = labels.foldRight((HNil, state))(combineLabelWithValue)
ic.head(resultTuple)
}
def getValues[WithValues <: HList, Values <: HList](withValues: WithValues)(
implicit mapper: Mapper.Aux[GetLabelValue.type, WithValues, Values]
): Values = {
withValues.map(GetLabelValue)
}
def getValuesFull[L <: HList, P, WithValues <: HList, Values <: HList](labels: L)(
implicit folder: RightFolder.Aux[L, (HNil.type, Map[String, Any]), combineLabelWithValue.type, P],
ic: IsComposite.Aux[P, WithValues, _],
mapper: Mapper.Aux[GetLabelValue.type, WithValues, Values]
): Values = {
val state = Map("a" -> 5, "b" -> "five")
val resultTuple = labels.foldRight((HNil, state))(combineLabelWithValue)
val withValues: WithValues = ic.head(resultTuple)
withValues.map(GetLabelValue)
}
The issue here is that you're ending up trying to map over an HList where the HNil is statically typed as HNil.type. This doesn't work in general—e.g. in a simplified case like this:
import shapeless._, ops.hlist.Mapper
val mapped1 = Mapper[poly.identity.type, HNil]
val mapped2 = Mapper[poly.identity.type, HNil.type]
mapped1 will compile, but mapped2 won't.
The trick is to change the HNil.type in your RightFolder types to HNil and then to call foldRight with HNil: HNil. This will make everything work just fine.
There are a few other suggestions I'd make (destructure the tuple in place of P instead of using IsComposite, skip the Aux on mapper and return mapper.Out instead of having a Value type parameter, etc.), but they're probably out of the scope of this question.

Shapeless deconstruct tuple in type parameter declaration

I am using a RightFolder that returns a Tuple2 and would like to return the _1 part. The first version rightFoldUntupled1 works fine but uses an additional IsComposite typeclass.
In the second version rightFoldUntupled2 I try to achieve the same without IsComposite by destructuring P as a Product2[_, Values]. That doesn't work any more - the compiler is happy to witness that P is a Product2[_,_], but as soon as I use a named type parameter for _1 it doesn't compile any more. Any idea why?
The full source (with sbt ready to ~run with implicit debug output) is here: https://github.com/mpollmeier/shapeless-playground/tree/f3cf049
case class Label[A](name: String)
val label1 = Label[Int]("a")
val labels = label1 :: HNil
object getValue extends Poly2 {
implicit def atLabel[A, B <: HList] = at[Label[A], (B, Map[String, Any])] {
case (label, (acc, values)) ⇒
(values(label.name).asInstanceOf[A] :: acc, values)
}
}
// compiles fine
val untupled1: Int :: HNil = rightFoldUntupled1(labels)
// [error] could not find implicit value for parameter folder:
// shapeless.ops.hlist.RightFolder.Aux[shapeless.::[Main.DestructureTupleTest.Label[Int],shapeless.HNil],
//(shapeless.HNil, Map[String,Any]),Main.DestructureTupleTest.getValue.type,P]
val untupled2: Int :: HNil = rightFoldUntupled2(labels)
def rightFoldUntupled1[L <: HList, P <: Product2[_, _], Values <: HList](labels: L)(
implicit folder: RightFolder.Aux[L, (HNil, Map[String, Any]), getValue.type, P],
ic: IsComposite.Aux[P, Values, _]
): Values = {
val state = Map("a" -> 5, "b" -> "five")
val resultTuple = labels.foldRight((HNil: HNil, state))(getValue)
ic.head(resultTuple)
}
def rightFoldUntupled2[L <: HList, Values, P <: Product2[_, Values]](labels: L)(
implicit folder: RightFolder.Aux[L, (HNil, Map[String, Any]), getValue.type, P]
): Values = {
val state = Map("a" -> 5, "b" -> "five")
val resultTuple = labels.foldRight((HNil: HNil, state))(getValue)
resultTuple._1
}
This should work same as rightFoldUntupled1:
def rightFoldUntupled2[L <: HList, Values, M](labels: L)(
implicit folder: RightFolder.Aux[L, (HNil, Map[String, Any]), getValue.type, (Values, M)]
): Values = {
val state = Map("a" -> 5, "b" -> "five")
val resultTuple = labels.foldRight((HNil: HNil, state))(getValue)
resultTuple._1
}