I try to implement some simple conversion to/from "JsValue" ( from play framework library) using "shapeless", as the beginning of my acquaintance with it. I know that such a thing is realized in a spray-json-shapeless library to work with sparay-json.
As I can see encoding goes well, but I got a problem with decoding.
This is my test:
"json de/serialization for N fields case class" should {
"work as expected" in {
val nFields = NFields(1, 2.3, "test", true, 3)
val jsValue = JsonEncoder[NFields].encode(nFields)
val result = JsonDecoder[NFields].decode(jsValue)
println(result.toString)
result must equal(nFields)
}
}
After encoding object to JsValue I get the correct JsObject like this: {JsObject#1686}{"f1":1,"f2":2.3,"f3":"test","f4":true,"f5":3} but after decoding, the result object is NFields(1,3.0,test,true,2) so the value of field “f2: Double” “2.3” falls into place of field “f5: Int” as “2” and the value of field “f5: Int” “3” falls into place of field “f2: Double” as “3.0” and of course test fails with “NFields(1,3.0,test,true,2) did not equal NFields(1,2.3,test,true,3)”.
My Hlist decoder:
implicit def hlistDecoder[K <: Symbol, H, T <: HList](
implicit
key: Witness.Aux[K],
hDecoder: Lazy[JsonDecoder[H]],
tDecoder: JsonDecoder[T]): JsonDecoder[FieldType[K,H] :: T] =
createInstance{
case obj: JsObject =>
val value= obj.value
field[K](hDecoder.value.decode(value.head._2)) :: tDecoder.decode(JsObject(obj.value.tail))
}
and Generic decoder:
implicit def genericDecoder[A, R](
implicit
gen: LabelledGeneric.Aux[A, R],
dec: Lazy[JsonDecoder[R]]
): JsonDecoder[A] = createInstance(a => gen.from(dec.value.decode(a)))
Where is the problem? Any hints appreciated! Thanks in advance.
Related
I have some code:
case class Name(name: String) extends StaticAnnotation
case class Example(#Name("baz") foo: String, bar: Int)
object nameAnnotationZip2 extends Poly2 {
implicit val newName: Case.Aux[Symbol, Some[Name], String] =
at((_, name) => name.value.name)
implicit val existingName: Case.Aux[Symbol, None.type, String] =
at((field, _) => field.name)
}
def renameRecord[
A,
Record <: HList,
Fields <: HList,
Annotations <: HList,
ZippedWith <: HList
](a: A)(implicit
generic: LabelledGeneric.Aux[A, Record],
fields: Keys.Aux[Record, Fields],
annotations: Annotations.Aux[Name, A, Annotations],
zippedWith: ZipWith.Aux[
Fields,
Annotations,
nameAnnotationZip2.type,
ZippedWith
]
) = {
val record = generic.to(a)
println(fields())
println(annotations())
zippedWith(fields(), annotations())
}
renameRecord(Example("qix", 5))
which attempts to create a record from an instance with renamed keys based on an annotation. unfortunately it always gives me a
could not find implicit value for parameter zippedWith: shapeless.ops.hlist.ZipWith.Aux[Fields,Annotations,nameAnnotationZip2.type,ZippedWith]
I haven't added the ZipWithKeys bit yet, but I believe I need to work around this issue first.
I believe I'm simply using Poly totally wrong somehow, but the compiler message isn't helping me find my error.
edit/part two:
I can work around by using Zip instead of ZipWith, converting to Sized, simply mapping the seq without Poly, and converting back to HList. what's the runtime performance difference between doing that and the above?
The type of Fields isn't Symbol, it's Symbol ## "foo". Case's generic parameters aren't variant so nameAnnotationZip2 isn't defined for the elements of Fields.
Adding a generic parameter to the cases fixes the issue
object nameAnnotationZip2 extends Poly2 {
implicit def newName[K <: Symbol]: Case.Aux[K, Some[Name], String] =
at((_, name) => name.value.name)
implicit def existingName[K <: Symbol]: Case.Aux[K, None.type, String] =
at((field, _) => field.name)
}
I'm reading in query parameters and converting them into a Map[Symbol, String]. I would like to add some type safety to these query parameters through a set of case classes.
These case classes will be different depending on the incoming http request, so this needs to support different case classes.
If the incoming query parameters don't match the defined case class the Parser should return None.
I have attempted to use shapeless to implement a generic parser. It work's if all of the parameters are of type String. But I need to support any type of query parameter.
I've tried to incorporate the implicit conversion logic seen in this post, but unable to get it working.
https://meta.plasm.us/posts/2015/11/08/type-classes-and-generic-derivation/ (new to shapeless)
Existing Parser (without string to type conversion):
class Parser[A] {
def from[R <: HList]
(m: Map[Symbol, String])
(implicit
gen: LabelledGeneric.Aux[A, R],
fromMap: FromMap[R]
): Option[A] = fromMap(m).map(gen.from)
}
object Parser {
def to[A]: Parser[A] = new Parser[A]
}
Tests describing issue:
class ParserSpec extends FlatSpec with Matchers {
private val sampleName: String = "Bob"
private val sampleVersion: Int = 1
//Partial Solution
case class QueryParams(name: String, version: String)
//Full Solution (not working)
case class QueryParams2(name: String, version: Int)
"A Parser" should "parse query parameters from a map with only string values" in {
val mapOfQueryParams = Map('name -> sampleName, 'version -> sampleVersion.toString)
val result = Parser.to[QueryParams].from(mapOfQueryParams)
result shouldBe 'defined
result.get.name shouldEqual sampleName
result.get.version shouldEqual sampleVersion.toString
}
it should "parse query parameters from a map with any type of value" in {
val mapOfQueryParams = Map('name -> sampleName, 'version -> sampleVersion.toString)
val result = Parser.to[QueryParams2].from(mapOfQueryParams)
//result is not defined as it's not able to convert a string to integer
result shouldBe 'defined
result.get.name shouldEqual sampleName
result.get.version shouldEqual sampleVersion
}
}
FromMap uses shapeless.Typeable to convert values to the expected type. So the easiest way to make your code work is to define an instance of Typeable to convert from String to Int (and additional Typeable instances for any value type, that appears in your case classes):
implicit val stringToInt: Typeable[Int] = new Typeable[Int] {
override def cast(t: Any): Option[Int] = t match {
case t: String => Try(t.toInt).toOption
case _ => Typeable.intTypeable.cast(t)
}
override def describe: String = "Int from String"
}
This is however not an intended use of Typeable, which is designed to confirm that a variable with type Any is already an instance of the expected type without any conversion. In other words it's intended to be a typesafe implementation of asInstanceOf, that can also work around type erasure.
For correctness you can define your own ReadFromMap typeclass, that uses your own Read typeclass for conversion from Strings to the expected types. Here is a simple implementation of the Read typeclass (assuming Scala 2.12):
import scala.util.Try
trait Read[T] {
def apply(string: String): Option[T]
}
object Read {
implicit val readString: Read[String] = Some(_)
implicit val readInt: Read[Int] = s => Try(s.toInt).toOption
// Add more implicits for other types in your case classes
}
And you can copy and adapt the implementation of FromMap to use this Read typeclass:
import shapeless._
import shapeless.labelled._
trait ReadFromMap[R <: HList] extends Serializable {
def apply(map: Map[Symbol, String]): Option[R]
}
object ReadFromMap {
implicit def hnil: ReadFromMap[HNil] = _ => Some(HNil)
implicit def hlist[K <: Symbol, V, T <: HList](implicit
keyWitness: Witness.Aux[K],
readValue: Read[V],
readRest: ReadFromMap[T]
): ReadFromMap[FieldType[K, V] :: T] = map => for {
value <- map.get(keyWitness.value)
converted <- readValue(value)
rest <- readRest(map)
} yield field[K](converted) :: rest
}
Then simply use this new typeclass in your Parser:
class Parser[A] {
def from[R <: HList]
(m: Map[Symbol, String])
(implicit
gen: LabelledGeneric.Aux[A, R],
fromMap: ReadFromMap[R]
): Option[A] = fromMap(m).map(gen.from)
}
I've got the following code to basically iterate through the fields of a case class and map them using a Poly to the same type and use ToList[HL, Out]
For simplicity we can assume the Poly does this:
object Schema extends Poly1 {
implicit def caseInt = at[Int](_ => "I'm an int")
implicit def caseString = at[String](_ => "Oh boy a string")
}
def infer[V1 <: Product, Out <: HList, MapperOut <: HList](v1: V1)(
implicit gen: Generic.Aux[V1, Out],
map: Mapper.Aux[Schema.type, Out, MapperOut],
to: ToList[MapperOut, String]
): List[String] = to (gen to v1 map Schema)
This is all very straightforward and works very well for simple scenarios:
case class Test(id: Int, text: String)
val list = infer(Test(2, "text"))
// List("I'm an int", "Oh boy a string")
Now going out to where the buses don't run:
class Automagical[T <: Product with Serializable : TypeTag] {
def instance: T
// The typetag bit is needed for something else
def convert: List[String] = infer(instance)
}
Sadly any call to above fails with:
could not find implicit value for parameter gen: shapeless.Generic.Aux[T,Out]
Bonus
How can I improve the infer method by not requiring an instance of T at all? Obviously type inference is fine, but I do need to somehow materialise a List[String] from an HList[Lub] and map over something.
Given I only ever care about the types, is it possible to derive a concrete instance of List[String] by only knowing the types to be poly mapped encoded as an HList?
Something like:
def infer[V1 <: Product, Out <: HList, MapperOut <: HList]()(
implicit gen: Generic.Aux[V1, Out],
map: Mapper.Aux[Schema.type, Out, MapperOut],
to: ToList[MapperOut, String]
): List[String] = {
// Magically build an HList with samples from its types.
// Or an alternative approach that gives the same outcome
val reifiedInstance = reify[Out]
to (reifiedInstance map Schema)
}
To ensure that convert has access to the implicit parameters for infer, they have to be present when the Automagical instance is created:
abstract class Automagical[T <: Product with Serializable : TypeTag,
Out <: HList, MapperOut <: HList]
(implicit
gen: Generic.Aux[T, Out],
map: Mapper.Aux[Schema.type, Out, MapperOut],
to: ToList[MapperOut, String]) {
def instance: T
// The typetag bit is needed for something else
def convert: List[String] = infer(instance)
}
I am trying to solve [this][1] question using Shapeless, in summary it's about converting a nested case class to Map[String,Any], here is the example:
case class Person(name:String, address:Address)
case class Address(street:String, zip:Int)
val p = Person("Tom", Address("Jefferson st", 10000))
The goal is to convert p to following:
Map("name" -> "Tom", "address" -> Map("street" -> "Jefferson st", "zip" -> 10000))
I am trying to do it using Shapeless LabelledGeneric, here is what I have so far:
import shapeless._
import record._, syntax.singleton._
import ops.record._
import shapeless.ops.record._
def writer[T,A<:HList,H<:HList](t:T)
(implicit lGeneric:LabelledGeneric.Aux[T,A],
kys:Keys.Aux[A,H],
vls:Values[A]) = {
val tGen = lGeneric.to(t)
val keys = Keys[lGeneric.Repr].apply
val values = Values[lGeneric.Repr].apply(tGen)
println(keys)
println(values)
}
I am trying to have a recursive writer check each value and try to make Map for each element in value. The above code works fine but when I want to iterate over values with a sample Poly, using the following code I got these errors.
values.map(identity)
//or
tGen.map(identity)
Error:(75, 19) could not find implicit value for parameter mapper: shapeless.ops.hlist.FlatMapper[shapeless.poly.identity.type,vls.Out]
values.flatMap(identity)
^
Error:(75, 19) not enough arguments for method flatMap: (implicit mapper: shapeless.ops.hlist.FlatMapper[shapeless.poly.identity.type,vls.Out])mapper.Out.
Unspecified value parameter mapper.
values.flatMap(identity)
^
I don't know why I'm getting that error. I also would be happy to know if there is an easier way to do the whole thing using Shapeless.
[1]: Scala macros for nested case classes to Map and other way around
Any time you want to perform an operation like flatMap on an HList whose type isn't statically known, you'll need to provide evidence (in the form of an implicit parameter) that the operation is actually available for that type. This is why the compiler is complaining about missing FlatMapper instances—it doesn't know how to flatMap(identity) over an arbitrary HList without them.
A cleaner way to accomplish this kind of thing would be to define a custom type class. Shapeless already provides a ToMap type class for records, and we can take it as a starting point, although it doesn't provide exactly what you're looking for (it doesn't work recursively on nested case classes).
We can write something like the following:
import shapeless._, labelled.FieldType, record._
trait ToMapRec[L <: HList] { def apply(l: L): Map[String, Any] }
Now we need to provide instances for three cases. The first case is the base case—the empty record—and it's handled by hnilToMapRec below.
The second case is the case where we know how to convert the tail of the record, and we know that the head is something that we can also recursively convert (hconsToMapRec0 here).
The final case is similar, but for heads that don't have ToMapRec instances (hconsToMapRec1). Note that we need to use a LowPriority trait to make sure that this instance is prioritized properly with respect to hconsToMapRec0—if we didn't, the two would have the same priority and we'd get errors about ambiguous instances.
trait LowPriorityToMapRec {
implicit def hconsToMapRec1[K <: Symbol, V, T <: HList](implicit
wit: Witness.Aux[K],
tmrT: ToMapRec[T]
): ToMapRec[FieldType[K, V] :: T] = new ToMapRec[FieldType[K, V] :: T] {
def apply(l: FieldType[K, V] :: T): Map[String, Any] =
tmrT(l.tail) + (wit.value.name -> l.head)
}
}
object ToMapRec extends LowPriorityToMapRec {
implicit val hnilToMapRec: ToMapRec[HNil] = new ToMapRec[HNil] {
def apply(l: HNil): Map[String, Any] = Map.empty
}
implicit def hconsToMapRec0[K <: Symbol, V, R <: HList, T <: HList](implicit
wit: Witness.Aux[K],
gen: LabelledGeneric.Aux[V, R],
tmrH: ToMapRec[R],
tmrT: ToMapRec[T]
): ToMapRec[FieldType[K, V] :: T] = new ToMapRec[FieldType[K, V] :: T] {
def apply(l: FieldType[K, V] :: T): Map[String, Any] =
tmrT(l.tail) + (wit.value.name -> tmrH(gen.to(l.head)))
}
}
Lastly we provide some syntax for convenience:
implicit class ToMapRecOps[A](val a: A) extends AnyVal {
def toMapRec[L <: HList](implicit
gen: LabelledGeneric.Aux[A, L],
tmr: ToMapRec[L]
): Map[String, Any] = tmr(gen.to(a))
}
And then we can demonstrate that it works:
scala> p.toMapRec
res0: Map[String,Any] = Map(address -> Map(zip -> 10000, street -> Jefferson st), name -> Tom)
Note that this won't work for types where the nested case classes are in a list, tuple, etc., but you could extend it to those cases pretty straightforwardly.
I have a problem with an approach provided by Travis Brown.
Some of nesting case classes are not converted to Map https://scalafiddle.io/sf/cia2jTa/0.
The answer was found here.
To correct the solution just wrap ToMapRec[T] in implicit parameters to Lazy[ToMapRec[T]]. Corrected fiddle https://scalafiddle.io/sf/cia2jTa/1
I am trying to make this work in a type-safe manner:
val rows = db.select( ID_COLUMN, STR("name"), INT("count") ). from("tablename") ......
for ( (id, name, count) <- rows ) {
//some code to use the individual values
}
I have not found, so far, another way of making this type-safe besides the shapeless.
I do know about slick and other ORMs, so please don't send me there.
HList seems to be the way to approach passing in a heterogeneous bunch of objects and get back a list of values with specific types.
I tried to do something like this :
trait ColDef [X] {
def colName
def valalue (m:Map[String, Object]):X
}
object XS extends ColDef[String]{
def colName = "s"
def value (m:Map[String, Object]) = m("s").asInstanceOf[String]
}
object XI extends ColDef[Integer]{
def colName = "i"
def value (m:Map[String, Object]) = m("i").asInstanceOf[Integer]
}
val xhl = XS::XI::HNil
val data:Map[String,Object] = Map(("s" ->"asdf"), ("i" -> new Integer(5)))
object p1 extends Poly1 {
implicit def getValue[T, S <% ColDef[T]] = at[S] (coldef => coldef.value(data) )
}
val res = xhl map p1
val (s, i) = res.tupled //this works, giving s:String, and i:Integer
//but following does not compile
def nextstep(hl : HList, data:Map[String,Object]) = {
hl map p1
}
Just to reiterate what is essential:
HList/Shapeless are likely candidates for solving the problem, but are not the goal of this exercise. My goal is to have the return type of a function correspond to the variable types and number of parameters passed in.
It would be ideal if the user of my little utility did not need to know about HList, but that is not a real requirement.
The essential part is having the type of the result match the type of params :
val param1 = new Conf[String]
val param2 = new Conf[Integer]
... etc ....
val res = function(param1, param2, param3)
More precisely payload types of the parameters above, so that the type of res is T(String, Integer, ....) .
Let me add another clarification. I want to create a method for arbitrary arity and avoid creating a function for each count of parameters. If I was ok with 22 methods, it would look something like this:
def f[A](a:ColDef[A]):(A)
def f[A,B](a:ColDef[A], b:ColDef[B]):(A,B)
def f[A,B,C](a:ColDef[A], b:ColDef[B],c:ColDef[C]):(A,B,C)
..... and so on
And then I would not need shapeless or HList as all the possible tuples would be explicitly defined.
Actually looking at those 3 signatures - making them 22 would take a bit of time, but would avoid shapeless dependency an their implementations would also be one-liners. May be I should just spend 30 minutes and do it manually (or with a little script).
Have it work with HList input and output
You just need some minor adjustments to the definition of nextstep:
def nextstep[L <: HList](hl: L, data: Map[String, Object])(implicit mapper: Mapper[p1.type, L]): mapper.Out = {
hl map p1
}
I made the exact type of the HList L a type argument, and I required the implicit needed by map (see map's definition).
Then you (or your user) can simply call
nextstep(XS :: XI :: HNil, data)
and they will get a String :: Integer :: HNil as return type. It works as expected for any HList of ColDef[...] (returning an HList of the result).
Have it work with a tuple as input (and output HList)
In order to have it return a tuple instead of an HList, you can define it this way:
import shapeless.ops.hlist.{Tupler, Mapper}
def nextstep[L <: HList, OutL <: HList, Out](hl: L, data: Map[String, Object])(implicit mapper: Mapper.Aux[p1.type, L, OutL], tupler: Tupler.Aux[OutL, Out]): Out = {
tupler(hl map p1)
}
nextstep(XS :: XI :: HNil, data) will then return a (String, Integer), and nextstep will return the rightly typed tuple in the general case.
Tuple as input and output
The final step to accept a tuple of ColDef as input and return a tuple as output looks like:
def nextstep[P, L <: HList, OutL <: HList, Out](c: P, data: Map[String, Object])(implicit gen: Generic.Aux[P, L], mapper: Mapper.Aux[p1.type, L, OutL], tupler: Tupler.Aux[OutL, Out]): Out = {
tupler(gen.to(c) map p1)
}
The logic here is very similar to the functions defined in shapeless.syntax.std.TupleOps: converting the tuple to an HList, processing the HList, convert the output to a tuple.