Using ClassTag with Variance - scala

I have a transfomer which is just a scala function, but needs ClassTag for interop with legacy code:
trait Transformer[F, T] extends (F => T) {
implicit def cF: ClassTag[F]
implicit def cT: ClassTag[T]
/**
* Append another transformer to this transformer
*/
def >=>[U: ClassTag](f: Transformer[T, U]): Transformer[F, U] = Transformer(this andThen f)
}
/** Helper to view a function as a transformer */
object Transformer {
def apply[F: ClassTag, T: ClassTag](f: F => T): Transformer[F, T] = new AbstractTransformer[F, T] {
override def apply(x: F): T = f(x)
}
}
abstract class AbstractTransformer[F, T](implicit override val cF: ClassTag[F], override val cT: ClassTag[T]) extends Transformer[F, T]
However, I would like to have the same variance in T and F as scala
Function1 which is
trait Transformer[-F, +T] extends (F => T)
However this results in
contravariant type F occurs in invariant position in type =>
scala.reflect.ClassTag[F] of method cF Transformer.scala
covariant type T occurs in invariant position in type =>
scala.reflect.ClassTag[T] of method cT Transformer.scala
Can I solve this problem somehow?

import scala.annotation.unchecked.uncheckedVariance
...
implicit def cF: ClassTag[F #uncheckedVariance]
implicit def cT: ClassTag[T #uncheckedVariance]
However, you need to be careful using this: there is a reason for the error! For example, consider
val t: Transformer[String, Object] = ...
val t1: Transformer[Object, String] = t // legal by variance
val c1 = t1.cF // the compiler thinks it's a ClassTag[Object], but really it's the ClassTag for String
val c2 = t1.cT // vice versa

Related

How to compare types of different, generic HLists?

So I have a user defined input of keys, and the user is also expected to provide serializers for these keys. I am trying to statically ensure that the serializers provided work with the user provided types. Example user usage would be something like:
case class Keys(key1: String, key2: Int)
val keys = Keys("Foo", 2)
val keyDescriptor = (StringSerializer, IntSerializer)
toSerializedKeys(keys, keyDescriptor)
but I would like something like this to fail at compile time:
val keys = Keys("Foo", 2)
val keyDescriptor = (StringSerializer, StringSerializer)
toSerializedKeys(keys, keyDescriptor)
To abstract over arity and attempt to provide compile time type matching on the keys/serializers I'm trying to use shapeless. My current solution works fine given explicit HLists. But as soon as I try to apply this to generic HLists things fall apart.
Current solution:
Assume we have the following user provided serializers:
trait SerializerFn[T] {
def toBytes(t: T): ByteBuffer
}
object StringSerializerFn extends SerializerFn[String] {
override def toBytes(t: String): ByteBuffer = ???
}
object IntSerializerFn extends SerializerFn[Int] {
override def toBytes(t: Int): ByteBuffer = ???
}
I currently create a type class, helper function, and instances to handle serialization of HLists:
trait Serializer[HA <: HList, HB <: HList] {
def toBufs(keys: HA)(fns: HB): List[ByteBuffer]
}
object Serializer extends SerializerInstances {
def toBufs[HA <: HList, HB <: HList](keys: HA)(fns: HB)(
implicit
serializer: Serializer[HA, HB]
): List[ByteBuffer] = {
serializer.toBufs(keys)(fns)
}
}
trait SerializerInstances {
implicit val hnilSerializer: Serializer[HNil, HNil] =
new Serializer[HNil, HNil] {
override def toBufs(keys: HNil)(fns: HNil): List[ByteBuffer] = Nil
}
implicit def hconsSerializer[H, Fn <: SerializerFn[H], TA <: HList, TB <: HList](
implicit
hconsSerializer: Serializer[TA, TB],
): Serializer[H :: TA, Fn :: TB] =
new Serializer[H :: TA, Fn :: TB] {
override def toBufs(keys: H :: TA)(fns: Fn :: TB): List[ByteBuffer] =
List(fns.head.toBytes(keys.head)) ::: hconsSerializer.toBufs(keys.tail)(fns.tail)
}
}
The static type checking I'm going for here is happening in hconsSerializer where we restrict the head of both HLists to be an H and SerializerFn[H], respectively.
And to test it, this compiles fine:
val testKeys = "3" :: "4" :: HNil
val testFns = StringSerializerFn :: StringSerializerFn :: HNil
val testSerializedKeys = Serializer.toBufs(testKeys)(testFns)
and as expected, this doesn't compile:
val testKeys = "3" :: "4" :: HNil
val testFns = StringSerializerFn :: IntSerializerFn :: HNil
val testSerializedKeys = Serializer.toBufs(testKeys)(testFns)
However, when trying to generalize this (so that we can actually use this in a constructor that we provide to users), things fall apart, (I've omitted the Generic conversion from the user provided Product to HList for brevity):
def toSerializedKeys[ReprA <: HList, ReprB <: HList](
keys: ReprA,
fns: ReprB
): List[ByteBuffer] =
Serializer.toBufs(keys)(fns)
For reference, intellij complains that the implicit Serializer can't be found, and explicitly passing hconsSerializer shows:
Required: Serializer[ ReprA, ReprB]
Found: Serializer[Nothing :: HNil, Nothing :: HNil]
and Scalac says:
[error] found : [H, Fn <: com.twitter.beam.io.manhattan.SerializerFn[H], TA <: shapeless.HList, TB <: shapeless.HList]com.twitter.beam.io.manhattan.Serializer[H :: TA,Fn :: TB]
[error] required: com.twitter.beam.io.manhattan.Serializer[ReprA,ReprB]
What kind of implicit evidence do I need to be able to perform this kind of recursive type checking on generic HLists? Is accomplishing something like this even possible? I'm new to Shapeless.
Also, if comparing HList types like this isn't possible. Another solution could be something that allowed the user to define individual implicit key serializers, instead of a full KeyDescriptor that we convert to an HList, e.g something like:
implicit val intSerializer = IntSerializerFn
implicit val strSerizlizer = StringSerializerFn
However, I tried something like this and then tried mapping over the Keys HList with a Poly1 that takes an implicit SerializerFn, but I ran into similar issues when trying to work with a generic HLists.
If you are using Scala3 (aka dotty), here is a simple solution using stdlib only for your purpose:
import scala.deriving.Mirror.ProductOf as PF
import java.nio.ByteBuffer
trait SerializerFn[T]:
def toBytes(t: T): ByteBuffer
object StringSerializerFn extends SerializerFn[String]:
override def toBytes(t: String): ByteBuffer = ???
object IntSerializerFn extends SerializerFn[Int]:
override def toBytes(t: Int): ByteBuffer = ???
case class SI(key1: String, key2: Int)
case class SII(key1: String, key2: Int, key3: Int)
def toSerializedKeys[K <: Product, S <: Tuple](keys: K, sers: S)
(using pf: PF[K], ev: S <:< Tuple.Map[pf.MirroredElemTypes, SerializerFn]): Unit = {}
// success
toSerializedKeys(SI("a", 1), (StringSerializerFn, IntSerializerFn))
toSerializedKeys(SII("a", 1, 2), (StringSerializerFn, IntSerializerFn, IntSerializerFn))
// compile fail
// toSerializedKeys(SI("a", 1), (StringSerializerFn, StringSerializerFn))
// toSerializedKeys(SII("a", 1, 2), (IntSerializerFn, IntSerializerFn, IntSerializerFn))
I guess you just missed evidence serializer: Serializer[ReprA, ReprB] in
def toSerializedKeys[ReprA <: HList, ReprB <: HList](
keys: ReprA,
fns: ReprB
): List[ByteBuffer] =
Serializer.toBufs(keys)(fns)
So just replace this with
def toSerializedKeys[ReprA <: HList, ReprB <: HList](
keys: ReprA,
fns: ReprB
)(implicit serializer: Serializer[ReprA, ReprB]): List[ByteBuffer] =
Serializer.toBufs(keys)(fns)
Then
case class Keys(key1: String, key2: Int)
val keys = Keys("Foo", 2)
val keyDescriptor = (StringSerializer, IntSerializer)
toSerializedKeys(keys, keyDescriptor)
compiles (as expected) and
val keys = Keys("Foo", 2)
val keyDescriptor = (StringSerializer, StringSerializer)
toSerializedKeys(keys, keyDescriptor)
doesn't (as expected).
Just noticed that Alec advised the same in comments.

Achieving Ad hoc polymorphism at function parameter level (mixing parameters of different type)

When I have a function in Scala:
def toString[T: Show](xs: T*): String = paths.map(_.show).mkString
And the following type class instances in scope:
implicit val showA: Show[MyTypeA]
implicit val showB: Show[MyTypeB]
I can use function toString in the following ways:
val a1: MyTypeA
val a2: MyTypeA
val stringA = toString(a1, a2)
val b1: MyTypeB
val b2: MyTypeB
val stringB = toString(b1, b2)
But I cannot call toString mixing parameters of type MyTypeA and MyTypeB:
// doesn't compile, T is inferred to be of type Any
toString(a1, b1)
Is it possible to redefine toString in such a way that it becomes possible to mix parameters of different types (but only for which a Show typeclass is available)?
Note that I am aware of the cats show interpolator which solves this specific example, but I'm looking for a solution which can be applied to different cases as well (e.g. toNumber).
I am also aware of circumventing the problem by calling .show on the parameters before passing them to the toString function, but I'm looking for a way to avoid this as it results in code duplication.
Example with shapeless:
object myToString extends ProductArgs { //ProductArgs allows changing variable number of arguments to HList
//polymorphic function to iterate over values of HList and change to a string using Show instances
object showMapper extends Poly1 {
implicit def caseShow[V](implicit show: Show[V]): Case.Aux[V, String] = {
at[V](v => show.show(v))
}
}
def applyProduct[ARepr <: HList](
l: ARepr
)(
implicit mapper: Mapper[showMapper.type, ARepr]
): String = l.map(showMapper).mkString("", "", "")
}
Now let's test it:
case class Test1(value: String)
case class Test2(value: String)
case class Test3(value: String)
implicit val show1: Show[Test1] = Show.show(_.value)
implicit val show2: Show[Test2] = Show.show(_.value)
println(myToString(Test1("a"), Test2("b"))) //"ab"
println(myToString(Test1("a"), Test2("b"), Test3("c"))) //won't compile since there's no instance of Show for Test3
By the way, I think toString is not the best name, because probably it can cause weird conflicts with toString from java.lang.Object.
If you don't want to mess with shapeless, another solution that comes to my mind is to just create functions with different arity:
def toString[A: Show](a: A): String = ???
def toString[A: Show, B: Show](a: A, b: B): String = ???
//etc
It's definitely cumbersome, but it might be the easiest way to solve your problem.
Here's one way to do it in Dotty (note that most of the Dotty-specific features used here are not necessary; they're just to make life easier, but being able to abstract over tuples of different arities is something you can't do (easily) in Scala 2):
opaque type Show[T] = T => String
opaque type ShowTuple[T <: Tuple] = T => String
object ShowTuple {
given ShowTuple[EmptyTuple] = _ => ""
given showTuple[H, T <: Tuple](using show: Show[H], showTail: ShowTuple[T]) as ShowTuple[H *: T] =
{ case h *: t => show(h) + "," + showTail(t) }
}
def multiToString[T <: Tuple](t: T)(using showTuple: ShowTuple[T]) =
showTuple(t)
It can be used like this:
class TypeA(val i: Int)
class TypeB(val s: String)
class TypeC(val b: Boolean)
given Show[TypeA] = t => s"TypeA(${t.i})"
given Show[TypeB] = t => s"TypeB(${t.s})"
given Show[TypeC] = t => s"TypeC(${t.b})"
println(multiToString((new TypeA(10), new TypeB("foo"), new TypeC(true))))
Using a type for which an implicit is not given fails:
class TypeD
multiToString((new TypeA(10), new TypeB("foo"), new TypeC(true), new TypeD))
Try it in Scastie
What is the type of paths?
If it's List[T] then there should be an implicit Show[T] in scope.
If it's List[Any] then there should be an implicit Show[Any] in scope.
If paths contains elements of different types and paths is not a List[Any] then paths shouldn't be a List[...] at all. It can be of type L <: HList. You can try
import shapeless.{HList, HNil, Poly1, Poly2}
import shapeless.ops.hlist.{LeftReducer, Mapper}
trait Show[T] {
def show(t: T): String
}
implicit class ShowOps[T](t: T) {
def show(implicit s: Show[T]): String = s.show(t)
}
object show extends Poly1 {
implicit def cse[T: Show]: Case.Aux[T, String] = at(_.show)
}
object concat extends Poly2 {
implicit def cse: Case.Aux[String, String, String] = at(_ + _)
}
def toString[L <: HList, L1 <: HList](xs: L)(implicit
mapper: Mapper.Aux[show.type, L, L1],
reducer: LeftReducer.Aux[L1, concat.type, String]
): String = xs.map(show).reduceLeft(concat)
type MyTypeA
type MyTypeB
implicit val showA: Show[MyTypeA] = ???
implicit val showB: Show[MyTypeB] = ???
val a1: MyTypeA = ???
val b1: MyTypeB = ???
toString(a1 :: b1 :: HNil)

Generic sum of two numeric expressions

I am writing a little toy language built on top of expressions. Here is some code to get the idea:
trait Expression[+T] {
def eval: T
}
case class Literal[+T](value: T) extends Expression[T] {
def eval = value
}
The parser builds a tree of expressions which are then evaluated by calling the eval method. Now I want to add a Sum expression that represents the sum of two other expressions:
case class Sum[+T: Numeric](left: Expression[T], right: Expression[T]) {
def eval = implicitly[Numeric[T]].plus(left.eval, right.eval)
}
This works fine if the left and right expression have the same type (as specified by the constructor). But naturally I would like it to work in the following case as well:
Sum(Literal(1.1), Literal(1))
This does not work because the compiler does not find an implicit argument of type Numeric[AnyVal], which makes sense.
I came up with the following code, using type bounds, to try to fix the issue:
case class Sum2[+T: Numeric, L <% T, R <% T](left: Expression[L], right: Expression[R]) extends Expression[T] {
def eval = implicitly[Numeric[T]].plus(left.eval, right.eval)
}
Now the compiler complains that left.eval and right.eval are not of type T. Casting to T using asInstanceOf[T] generates more compiler errors because of ambiguous implicit arguments.
What is the proper way to achieve this?
As it was pointed in the comments, the fact that there is safe conversion from Int to Double for your operation is not enough for the compiler to be able prove that this conversion is valid in all relevant contexts. I'm not aware of any simpler way to achieve what you want than this code (see also online):
trait Expression[+T] {
def eval: T
}
trait TypeConverter[S, T] {
def convert(value: S): T
}
trait TypeConverterLowPriority {
implicit def compose[A, B, C](implicit aToB: TypeConverter[A, B], bToC: TypeConverter[B, C]): TypeConverter[A, C] = new TypeConverter.TypeConverterImpl(a => bToC.convert(aToB.convert(a)))
}
object TypeConverter extends TypeConverterLowPriority {
class TypeConverterImpl[S, T](f: S => T) extends TypeConverter[S, T] {
override def convert(value: S): T = f(value)
}
def sameType[T]: TypeConverter[T, T] = new TypeConverterImpl(identity)
implicit val intToDouble: TypeConverter[Int, Double] = new TypeConverterImpl(_.toDouble)
implicit val shortToInt: TypeConverter[Short, Int] = new TypeConverterImpl(_.toInt)
// add more "primitive" type conversions here
}
case class Literal[+T](value: T) extends Expression[T] {
def eval = value
}
trait BinaryOpImpl[A, B, R] {
protected val numericR: Numeric[R]
protected val aToR: TypeConverter[A, R]
protected val bToR: TypeConverter[B, R]
final def eval(left: A, right: B): R = evalImpl(aToR.convert(left), bToR.convert(right))
protected def evalImpl(left: R, right: R): R
}
trait BinaryOpImplCompanionLowPriority[Ops[_, _, _]] {
protected def build[A, B, R](numericR: Numeric[R], aToR: TypeConverter[A, R], bToR: TypeConverter[B, R]): Ops[A, B, R]
implicit def castLeftToRight[L, R: Numeric](implicit tcl: TypeConverter[L, R]): Ops[L, R, R] = build(implicitly[Numeric[R]], tcl, TypeConverter.sameType)
implicit def castRightToLeft[L: Numeric, R](implicit tcr: TypeConverter[R, L]): Ops[L, R, L] = build(implicitly[Numeric[L]], TypeConverter.sameType, tcr)
}
trait BinaryOpImplCompanion[Ops[_, _, _]] extends BinaryOpImplCompanionLowPriority[Ops] {
implicit def sameType[T: Numeric]: Ops[T, T, T] = build(implicitly[Numeric[T]], TypeConverter.sameType, TypeConverter.sameType)
}
class SumImpl[A, B, R](val numericR: Numeric[R], val aToR: TypeConverter[A, R], val bToR: TypeConverter[B, R]) extends BinaryOpImpl[A, B, R] {
override protected def evalImpl(left: R, right: R): R = numericR.plus(left, right)
}
object SumImpl extends BinaryOpImplCompanion[SumImpl] {
override protected def build[A, B, R](numericR: Numeric[R], aToR: TypeConverter[A, R], bToR: TypeConverter[B, R]): SumImpl[A, B, R] = new SumImpl(numericR, aToR, bToR)
}
case class Sum[+T, L, R](left: Expression[L], right: Expression[R])(implicit impl: SumImpl[L, R, T]) extends Expression[T] {
def eval = impl.eval(left.eval, right.eval)
}
usage example:
def test(): Unit = {
println(Sum(Literal(3), Literal(1)).eval)
println(Sum(Literal(1.1), Literal(1)).eval)
println(Sum(Literal(1), Literal(1.1)).eval)
println(Sum(Literal[Short](1), Literal(1.12)).eval) // composite conversion Short -> Int -> Double
}
Essentially the idea is to have one implicit variable that encapsulates all 3 relevant types instead of having 3 separate implicits. So the code complies if the compiler can build one composite evidence for a triplet LeftArgType-RightArgType-ResultType.
the problem specifically is that Sum(Literal(1.1), Literal(1)) has a Literal[Double] on the left and a Literal[Int] on the right. The LUB of Int and Double is indeed AnyVal as you have seen.
https://scalafiddle.io/sf/ALM9urR/1
works perfectly fine. I also think this is good behavior because adding different types can be a bit iffy but else you could introduce an implicit that lets you do the necessary conversions.

Scala: upper bound [T <: AnyRef] does not allow to return [AnyRef]

I'm trying to express the following idea:
Function caseClassFields should return an array of (String, T) pairs, by processing a case class.
I put upper bound for T, expecting that it should be a subtype of AnyRef or AnyRef itself.
Here is a function:
def caseClassFields[T <: AnyRef](obj: AnyRef): Array[(String, T)] = {
val metaClass = obj.getClass
metaClass.getDeclaredFields.map {
field => {
field.setAccessible(true)
(field.getName, field.get(obj))
}
}
}
But unfortunately I get following error:
Expression of type Array[(String, AnyRef)] doesn't conform to expected type Array[(String, T)]
How to fix this?
Doing what you want with reflection and keeping type safety are orthogonal requirements. But shapeless, a library for generic derivation, can do what you want and still keep you type safe.
Here's a short example using shapeless to get you started.
We first define our algebra:
sealed trait ValidatableField
case class ValidatableString(value: Boolean)
extends ValidatableField
case class ValidatableInt(value: Boolean) extends ValidatableField
case class ValidatableRecord(fields: List[(String, ValidatableField)])
extends ValidatableField
Now we define our validator trait:
trait Validator[T] {
def validate(value: T): ValidatableField
}
trait RecordValidator[T] extends Validator[T] {
def validate(value: T): ValidatableRecord
}
Now lets define, for the sake of the example, validation on Int and String:
implicit val intValidator = new Validator[Int] {
override def validate(t: Int): ValidatableField = ValidatableInt(t > 42)
}
implicit val stringValidator = new Validator[String] {
override def validate(t: String): ValidatableField = ValidatableString(t.length < 42)
}
Now we define a generic implementation for HList which will cover our ValidatableRecord which is the generic representation of our case class:
implicit val hnilEncoder: RecordValidator[HNil] = new RecordValidator[HNil] {
override def validate(value: HNil): ValidatableRecord = ValidatableRecord(Nil)
}
implicit def hlistValidator[K <: Symbol, H, T <: HList](
implicit witness: Witness.Aux[K],
hEncoder: Lazy[Validator[H]],
tEncoder: RecordValidator[T]
): RecordValidator[FieldType[K, H] :: T] = {
val fieldName = witness.value.name
new RecordValidator[::[FieldType[K, H], T]] {
override def validate(value: ::[FieldType[K, H], T]): ValidatableRecord = {
val head = hEncoder.value.validate(value.head)
val tail = tEncoder.validate(value.tail)
ValidatableRecord((fieldName, head) :: tail.fields)
}
}
}
implicit def genericEncoder[A, H <: HList](
implicit generic: LabelledGeneric.Aux[A, H],
hEncoder: Lazy[RecordValidator[H]]): Validator[A] = {
new RecordValidator[A] {
override def validate(value: A): ValidatableRecord =
hEncoder.value.validate(generic.to(value))
}
}
With this much code, we can now validate any case class which has a String and Int field in it, and it is trivial to add other validator for more primitives:
object Test {
def main(args: Array[String]): Unit = {
case class Foo(s: String, i: Int)
val foo = Foo("hello!", 42)
println(Validator[Foo].validate(foo))
}
}
Yields:
ValidatableRecord(List((s,ValidatableString(true)), (i,ValidatableInt(false))))
I know this can be overwhelming a bit, but David Gurnells "Guide To Shapeless" is a great place to get started.
The reason is field.get(obj) returns AnyRef while your return type is T. Therefore, you need to convert it into T. However, I don't see any use of Generic type T in your code, so you can simply change the return type to Array[(String, AnyRef)].
def caseClassFields[T <: AnyRef](obj: AnyRef): Array[(String, AnyRef)]
However, if you insist to use Generic, you need to convert field.get(obj) to type T. Note, that you might get exception in case of invalid type while converting to type T.
def caseClassFields[T <: AnyRef](obj: AnyRef): Array[(String, T)] = {
val metaClass = obj.getClass
metaClass.getDeclaredFields.map {
field => {
field.setAccessible(true)
(field.getName, field.get(obj).asInstanceOf[T])
}
}
}
case class Foo(name:String)
val result:Array[(String, String)] = caseClassFields[String](Foo("bar"))
As discussed in the comments you probably want to use shapeless, but to elaborate.
The method field.get() isnt related to the type parameter. Normally you would use a type parameter like this
def caseClassFields[T <: AnyRef](obj: T): Array[(String, T)] = ???
or this ..
def caseClassFields[T <: AnyRef](obj: Container[T]): Array[(String, T)] = ???
If there was some link between field.get and T it could work, but the relationship would need to be proven to the compiler. As the T can be anything the compiler cant prove anything about it.
I can strongly recommend this book the 'Type Astronaut’s Guide to Shapeless' as a intro to the topic.
http://underscore.io/books/shapeless-guide/
https://github.com/milessabin/shapeless

How to shapeless case classes with attributes and typeclasses?

I am currently implementing a library to serialize and deserialize to and from XML-RPC messages. It's almost done but now I am trying to remove the boilerplate of my current asProduct method using Shapeless. My current code:
trait Serializer[T] {
def serialize(value: T): NodeSeq
}
trait Deserializer[T] {
type Deserialized[T] = Validation[AnyErrors, T]
type AnyErrors = NonEmptyList[AnyError]
def deserialize(from: NodeSeq): Deserialized[T]
}
trait Datatype[T] extends Serializer[T] with Deserializer[T]
// Example of asProduct, there are 20 more methods like this, from arity 1 to 22
def asProduct2[S, T1: Datatype, T2: Datatype](apply: (T1, T2) => S)(unapply: S => Product2[T1, T2]) = new Datatype[S] {
override def serialize(value: S): NodeSeq = {
val params = unapply(value)
val b = toXmlrpc(params._1) ++ toXmlrpc(params._2)
b.theSeq
}
// Using scalaz
override def deserialize(from: NodeSeq): Deserialized[S] = (
fromXmlrpc[T1](from(0)) |#| fromXmlrpc[T2](from(1))
) {apply}
}
My goal is to allow the user of my library to serialize/deserialize case classes without force him to write boilerplate code. Currently, you have to declare the case class and an implicit val using the aforementioned asProduct method to have a Datatype instance in context. This implicit is used in the following code:
def toXmlrpc[T](datatype: T)(implicit serializer: Serializer[T]): NodeSeq =
serializer.serialize(datatype)
def fromXmlrpc[T](value: NodeSeq)(implicit deserializer: Deserializer[T]): Deserialized[T] =
deserializer.deserialize(value)
This is the classic strategy of serializing and deserializing using type classes.
At this moment, I have grasped how to convert from case classes to HList via Generic or LabelledGeneric. The problem is once I have this conversion done how I can call the methods fromXmlrpc and toXmlrpc as in the asProduct2 example. I don't have any information about the types of the attributes in the case class and, therefore, the compiler cannot find any implicit that satisfy fromXmlrpc and toXmlrpc. I need a way to constrain that all the elements of a HList have an implicit Datatype in context.
As I am a beginner with Shapeless, I would like to know what's the best way of getting this functionality. I have some insights but I definitely have no idea of how to get it done using Shapeless. The ideal would be to have a way to get the type from a given attribute of the case class and pass this type explicitly to fromXmlrpc and toXmlrpc. I imagine that this is not how it can be done.
First, you need to write generic serializers for HList. That is, you need to specify how to serialize H :: T and HNil:
implicit def hconsDatatype[H, T <: HList](implicit hd: Datatype[H],
td: Datatype[T]): Datatype[H :: T] =
new Datatype[H :: T] {
override def serialize(value: H :: T): NodeSeq = value match {
case h :: t =>
val sh = hd.serialize(h)
val st = td.serialize(t)
(sh ++ st).theSeq
}
override def deserialize(from: NodeSeq): Deserialized[H :: T] =
(hd.deserialize(from.head) |#| td.deserialize(from.tail)) {
(h, t) => h :: t
}
}
implicit val hnilDatatype: Datatype[HNil] =
new Datatype[HNil] {
override def serialize(value: HNil): NodeSeq = NodeSeq()
override def deserialize(from: NodeSeq): Deserialized[HNil] =
Success(HNil)
}
Then you can define a generic serializer for any type which can be deconstructed via Generic:
implicit def genericDatatype[T, R](implicit gen: Generic.Aux[T, R],
rd: Lazy[Datatype[R]]): Datatype[T] =
new Datatype[T] {
override def serialize(value: T): NodeSeq =
rd.value.serialize(gen.to(value))
override def deserialize(from: NodeSeq): Deserialized[T] =
rd.value.deserialize(from).map(rd.from)
}
Note that I had to use Lazy because otherwise this code would break the implicit resolution process if you have nested case classes. If you get "diverging implicit expansion" errors, you could try adding Lazy to implicit parameters in hconsDatatype and hnilDatatype as well.
This works because Generic.Aux[T, R] links the arbitrary product-like type T and a HList type R. For example, for this case class
case class A(x: Int, y: String)
shapeless will generate a Generic instance of type
Generic.Aux[A, Int :: String :: HNil]
Consequently, you can delegate the serialization to recursively defined Datatypes for HList, converting the data to HList with Generic first. Deserialization works similarly but in reverse - first the serialized form is read to HList and then this HList is converted to the actual data with Generic.
It is possible that I made several mistakes in the usage of NodeSeq API above but I guess it conveys the general idea.
If you want to use LabelledGeneric, the code would become slightly more complex, and even more so if you want to handle sealed trait hierarchies which are represented with Coproducts.
I'm using shapeless to provide generic serialization mechanism in my library, picopickle. I'm not aware of any other library which does this with shapeless. You can try and find some examples how shapeless could be used in this library, but the code there is somewhat complex. There is also an example among shapeless examples, namely S-expressions.
Vladimir's answer is great and should be the accepted one, but it's also possible to do this a little more nicely with Shapeless's TypeClass machinery. Given the following setup:
import scala.xml.NodeSeq
import scalaz._, Scalaz._
trait Serializer[T] {
def serialize(value: T): NodeSeq
}
trait Deserializer[T] {
type Deserialized[T] = Validation[AnyErrors, T]
type AnyError = Throwable
type AnyErrors = NonEmptyList[AnyError]
def deserialize(from: NodeSeq): Deserialized[T]
}
trait Datatype[T] extends Serializer[T] with Deserializer[T]
We can write this:
import shapeless._
object Datatype extends ProductTypeClassCompanion[Datatype] {
object typeClass extends ProductTypeClass[Datatype] {
def emptyProduct: Datatype[HNil] = new Datatype[HNil] {
def serialize(value: HNil): NodeSeq = Nil
def deserialize(from: NodeSeq): Deserialized[HNil] = HNil.successNel
}
def product[H, T <: HList](
dh: Datatype[H],
dt: Datatype[T]
): Datatype[H :: T] = new Datatype[H :: T] {
def serialize(value: H :: T): NodeSeq =
dh.serialize(value.head) ++ dt.serialize(value.tail)
def deserialize(from: NodeSeq): Deserialized[H :: T] =
(dh.deserialize(from.head) |#| dt.deserialize(from.tail))(_ :: _)
}
def project[F, G](
instance: => Datatype[G],
to: F => G,
from: G => F
): Datatype[F] = new Datatype[F] {
def serialize(value: F): NodeSeq = instance.serialize(to(value))
def deserialize(nodes: NodeSeq): Deserialized[F] =
instance.deserialize(nodes).map(from)
}
}
}
Be sure to define these all together so they'll be properly companioned.
Then if we have a case class:
case class Foo(bar: String, baz: String)
And instances for the types of the case class members (in this case just String):
implicit object DatatypeString extends Datatype[String] {
def serialize(value: String) = <s>{value}</s>
def deserialize(from: NodeSeq) = from match {
case <s>{value}</s> => value.text.successNel
case _ => new RuntimeException("Bad string XML").failureNel
}
}
We automatically get a derived instance for Foo:
scala> case class Foo(bar: String, baz: String)
defined class Foo
scala> val fooDatatype = implicitly[Datatype[Foo]]
fooDatatype: Datatype[Foo] = Datatype$typeClass$$anon$3#2e84026b
scala> val xml = fooDatatype.serialize(Foo("AAA", "zzz"))
xml: scala.xml.NodeSeq = NodeSeq(<s>AAA</s>, <s>zzz</s>)
scala> fooDatatype.deserialize(xml)
res1: fooDatatype.Deserialized[Foo] = Success(Foo(AAA,zzz))
This works about the same as Vladimir's solution, but it lets Shapeless abstract some of the boring boilerplate of type class instance derivation so you don't have to get your hands dirty with Generic.