How to write custom circe codec for Map[String, Any] - scala

Is it possible to write custom Decoder for Map[String, Any] using circe? i've found this, but it's only conversion to Json:
def mapToJson(map: Map[String, Any]): Json =
map.mapValues(anyToJson).asJson
def anyToJson(any: Any): Json = any match {
case n: Int => n.asJson
case n: Long => n.asJson
case n: Double => n.asJson
case s: String => s.asJson
case true => true.asJson
case false => false.asJson
case null | None => None.asJson
case list: List[_] => list.map(anyToJson).asJson
case list: Vector[_] => list.map(anyToJson).asJson
case Some(any) => anyToJson(any)
case map: Map[String, Any] => mapToJson(map)
}

This Circe decoder will convert its Json into Map[String, Any]:
import io.circe.JavaDecoder._
implicit val objDecoder: Decoder[Any] = {
case x: HCursor if x.value.isObject =>
x.value.as[java.util.Map[String, Any]]
case x: HCursor if x.value.isString =>
x.value.as[String]
case x: HCursor if x.value.isBoolean =>
x.value.as[Boolean]
case x: HCursor if x.value.isArray =>
x.value.as[java.util.List[Any]]
case x: HCursor if x.value.isNumber =>
x.value.as[Double]
case x: HCursor if x.value.isNull =>
x.value.as[Unit]}
import io.circe.parser._
parse("""{"foo": 1, "bar": null, "baz": {"a" : "hi", "b" : true, "dogs" : [{ "name" : "Rover", "age" : 4}, { "name" : "Fido", "age" : 5 }] } }""")
.getOrElse(Json.Null)
.as[Map[String, _]]. // <-- this is the call to convert from Circe Json to the Java Map
It would be nicer to case match on the cursor value, but those classes are not public.
Note that I encoded null as Unit, which isn't quite right -- Scala and null don't play nicely and you may need to tailor your implementation to your use case. I actually omit that case entirely from my own implementation because I'm decoding Json that was encoded from Scala instances.
You also need to create a custom Decoder in io.circe for Java Map and List. I did the following, which uses package-private Circe code, making it concise but also vulnerable to changes in Circe and forcing you to have your own io.circe package.
package io.circe
import java.util.{List => JavaList, Map => JavaMap}
import scala.collection.mutable
import scala.collection.immutable.{List => ScalaList, Map => ScalaMap}
import scala.jdk.CollectionConverters.{MapHasAsJava, SeqHasAsJava}
object JavaDecoder {
implicit final def decodeJavaList[A](implicit decodeA: Decoder[A]): Decoder[JavaList[A]] = new SeqDecoder[A, JavaList](decodeA) {
final protected def createBuilder(): mutable.Builder[A, JavaList[A]] =
ScalaList.newBuilder[A].mapResult(_.asJava)
}
implicit final def decodeJavaMap[K, V](implicit
decodeK: KeyDecoder[K],
decodeV: Decoder[V]
): Decoder[JavaMap[K, V]] = new JavaMapDecoder[K, V, JavaMap](decodeK, decodeV) {
final protected def createBuilder(): mutable.Builder[(K, V), JavaMap[K, V]] =
ScalaMap.newBuilder[K, V].mapResult(_.asJava)
}
}
package io.circe
import scala.collection.{Map => ScalaMap}
import scala.collection.mutable
import scala.jdk.CollectionConverters.MapHasAsJava
abstract class JavaMapDecoder[K, V, M[K, V] <: java.util.Map[K, V]](
decodeK: KeyDecoder[K],
decodeV: Decoder[V]
) extends Decoder[M[K, V]] {
private val mapDecoder = new MapDecoder[K, V, ScalaMap](decodeK, decodeV) {
override protected def createBuilder(): mutable.Builder[(K, V), ScalaMap[K, V]] =
ScalaMap.newBuilder[K, V]
}
override def apply(c: io.circe.HCursor): io.circe.Decoder.Result[M[K,V]] =
mapDecoder.apply(c).map(_.asJava.asInstanceOf[M[K, V]])
}
Fwiw, I'm already thinking of submitting this as a PR to Circe.

import io.circe.syntax.EncoderOps
import io.circe.{Encoder, Json}
case class Person(name: String, age: Int)
object Person {
implicit val decoder: io.circe.Decoder[Person] = io.circe.generic.semiauto.deriveDecoder
implicit val encoder: io.circe.Encoder[Person] = io.circe.generic.semiauto.deriveEncoder
}
case class Home(area: Int)
object Home {
implicit val decoder: io.circe.Decoder[Home] = io.circe.generic.semiauto.deriveDecoder
implicit val encoder: io.circe.Encoder[Home] = io.circe.generic.semiauto.deriveEncoder
}
def jsonPrinter[A](obj: A)(implicit encoder: Encoder[A]): Json =
obj.asJson
jsonPrinter(Person("Eminem", 30))
jsonPrinter(Home(300))
This is done with generics, I hope it helps

Related

Codec for coproducts with discriminator

We're trying to have circe codecs for shapeless coproducts whith a class name as the discriminator :
object CoproductCodecWrong extends App {
import io.circe.Codec
import shapeless.{ :+:, CNil, Coproduct }
import io.circe.shapes._
import io.circe.literal._
case class A()
case class B(a: String)
type AB = A :+: B :+: CNil
case class H(ab: AB)
implicit val encodeA: Codec[A] = io.circe.generic.semiauto.deriveCodec
implicit val encodeB: Codec[B] = io.circe.generic.semiauto.deriveCodec
implicit val hCodec: Codec.AsObject[H] = io.circe.generic.semiauto.deriveCodec[H]
val hA: H = H(Coproduct[AB](A()))
val hB: H = H(Coproduct[AB](B("q")))
}
It doesn't work because by default the encoder doesn't produce any information that allows to distinguish both cases:
println("hA:\n" + hCodec.apply(hA).toString())
println("hB:\n" + hCodec.apply(hB).toString())
...
hA:
{
"ab" : {
}
}
hB:
{
"ab" : {
"a" : "q"
}
}
Decoding:
println(hCodec.decodeJson(json"""{"ab" : {}}"""))
println(hCodec.decodeJson(json"""{"ab" : {"a" : "q"}}"""))
...
Right(H(Inl(A())))
Right(H(Inl(A())))
It's always A that is found, which is very dangerous as it's working seemingly correctly but should be failing.
We've found a solution using Unions:
import io.circe.{ CursorOp, Decoder, DecodingFailure, Encoder }
import shapeless.ops.coproduct.Folder
import shapeless.ops.union.Values
import shapeless.union.Union
import shapeless.Poly1
type ABL = Union.`'A->A,'B->B`.T
object ABPoly extends Poly1 {
implicit val atA: Case.Aux[A, ABL] = at[A] { z =>
Union[ABL](A = z)
}
implicit val atB: Case.Aux[B, ABL] = at[B] { z =>
Union[ABL](B = z)
}
}
def coproductToLabeledCoproduct[Co <: Coproduct, Labeled <: Coproduct](t: Poly1)(
implicit E: Encoder[Labeled],
D: Decoder[Labeled],
V: Values.Aux[Labeled, Co],
F: Folder.Aux[t.type, Co, Labeled]
): Codec[Co] = {
import io.circe.syntax._
Codec.from(
Decoder.instance { input =>
input.focus
.map(_.as[Labeled].map(x => shapeless.union.unionOps(x).values: Co))
.getOrElse(Left(DecodingFailure("Should be available", { List(CursorOp.DownField("coproduct instance")) })))
},
Encoder.instance(a => a.fold(t).asJson)
)
}
implicit val abToABL: Codec[AB] = coproductToLabeledCoproduct[AB, ABL](ABPoly)
implicit val hCodec: Codec.AsObject[H] = io.circe.generic.semiauto.deriveCodec[H]
The result of encoding/decoding is correct now:
hA:
{
"ab" : {
"A" : {
}
}
}
hB:
{
"ab" : {
"B" : {
"a" : "q"
}
}
}
Right(H(Inl(A())))
Right(H(Inr(Inl(B(q)))))
But it's a bit complex, and hard to maintain for multiple classes.
Is there a better way?
EDIT:
circe-generic-extras doesn't work as it doesn't have a support for coproducts, only for sealed traits and enums. This gives the same result without a discriminator:
implicit val configuration: Configuration = Configuration.default.withDiscriminator("tag")
implicit val encodeA: Codec[A] = io.circe.generic.extras.semiauto.deriveConfiguredCodec
implicit val encodeB: Codec[B] = io.circe.generic.extras.semiauto.deriveConfiguredCodec
implicit val hCodec: Codec.AsObject[H] = io.circe.generic.extras.semiauto.deriveConfiguredCodec[H]
If you are using HLists and Coproducts directly then I guess something is broken in the code that requires that but I would probably use something like:
import shapeless._
import io.circe._
import io.circe.syntax._
class Discriminator(val fieldName: String, val classNameMapper: String => String)
implicit def cnilCodec: Codec[CNil] =
Codec.from(Decoder.instance[CNil](_ => Left(DecodingFailure("should never happen", Nil))), Encoder.instance(_ => ???))
implicit def coproductCodec[LeftLabel, LeftType: Codec, Right <: Coproduct: Codec](
implicit label: Witness.Aux[LeftLabel],
discriminator: Discriminator
): Codec[FieldType[LeftLabel, LeftType] :+: Right] = {
val thisLabel = discriminator.classNameMapper(label.value.toString)
val decoder = Decoder.instance[FieldType[LeftLabel, LeftType] :+: Right] { hcursor =>
hcursor
.get[String](discriminator.fieldName)
.leftMap(_.copy(message = s"Expected discriminator key ${discriminator.fieldName}"))
.flatMap { thisLabel =>
if (thisLabel == thisLabel) Decoder[LeftType].apply(hcursor).map(_.asInstanceOf[FieldType[LeftLabel, LeftType]]).map(Inl(_))
else Decoder[Right].apply(hcursor).map(Inr(_))
}
}
val encoder = Encoder.instance[FieldType[LeftLabel, LeftType] :+: Right] {
case Inl(value) => (value: LeftType).asJson.deepMerge(Json.fromFields(Seq(discriminator.fieldName -> thisLabel.asJson)))
case Inr(tail) => tail.asJson
}
Codec.from(decoder, encoder)
}
Whether it's simpler or more complicated it's a matter of taste.
For me personally this would be a red flag in deciding whether or not use the library in my code. Especially considering that you are forced to use shapeless directly with Circe, which supports sealed hierarchies out of the box.
I would use Circe Generic Extras with a sealed hierarchy.
It allows defining discriminator field name and format, providing support for auto- and semiauto derivation as well as macro annotations (#ConfiguredJsonCodec instead of #JsonCodec).
And then I would just translate this into this sad internal representation with:
def rewriteForCoproduct[A: Codec, ARepr <: Coproduct](implicit gen: LabelledGeneric.Aux[A, ARepr]): Codec[ARepr] =
Codec.from(Codec[A].map[ARepr](gen.to), Codec[A].contramap[ARepr](gen.from))

akka http (un)marshall traits

Let's assume the following Code:
sealed trait Action {
def run(): Boolean
}
case class SimpleAction(parameter: String) extends Actions {
// some impl
}
case class ExtendedAction(parameter1: String, parameter2: String) extends Actions {
// some impl
}
Now I want to define a webservice where one can retrieve the Actions. How can I Marshall the Action as it's just the trait and no specific Type?
I have found this https://github.com/spray/spray-json#providing-jsonformats-for-other-types in the Docs. Is there any simpler way to achieve this than using this approach mixed with pattern matching?
import spray.json._
import DefaultJsonProtocol._
implicit val simpleActionFormat = jsonFormat1(SimpleAction)
implicit val extendedActionFormat = jsonFormat2(ExtendedAction)
implicit val actionFormat1 = new JsonFormat[Action] {
override def write(obj: Action): JsValue = obj match {
case a: SimpleAction => JsObject("type" -> "simple".toJson, "value" -> a.toJson)
case b: ExtendedAction => JsObject("type" -> "extended".toJson, "value" -> b.toJson)
}
override def read(json: JsValue): Action = json.asJsObject.getFields("type", "value") match {
case Seq(JsString("simple"), js) => js.convertTo[SimpleAction]
case Seq(JsString("extended"), js) => js.convertTo[ExtendedAction]
case _ => throw new RuntimeException(s"Invalid json format: $json")
}
}
Or if you only care about converting Actions to json, then simply:
implicit val simpleActionFormat = jsonFormat1(SimpleAction)
implicit val extendedActionFormat = jsonFormat2(ExtendedAction)
implicit val actionFormat = lift(new JsonWriter[Action] {
override def write(obj: Action): JsValue = obj match {
case a: SimpleAction => a.toJson
case b: ExtendedAction => b.toJson
}
})

How to apply a function on each field of a case class

Let's consider a classification problem :
object Classify extends App {
type Tag = String
type Classifier[A] = A => Set[Tag]
case class Model(a: Int, b: String, c: String, d: String)
def aClassifier : Classifier[Int] = _ => Set("A", "a")
def bClassifier : Classifier[String] = _ => Set("B")
def cClassifier : Classifier[String] = _ => Set("C")
def modelClassifier : Classifier[Model] = {
m => aClassifier(m.a) ++ bClassifier(m.b) ++ cClassifier(m.c)
}
println(modelClassifier(Model(1,"b", "c", "d")))
}
Is there a smarter way to implement modelClassifier using scalaz ?
As an idea, consider this code:
for (i <- 0 until model.productArity) yield {
val fieldValue = model.productElement(i)
fieldValue match {
case x: Int => //use integer classifier
case s: String => //use string classifier
case _ =>
}
}
scalaz library hasn't any macro case class introspection by design, but shapeless has
Consider such definitions:
import shapeless._
import shapeless.tag._
import shapeless.labelled._
trait Omit
val omit = tag[Omit]
case class Model(a: Int, b: String, c: String, d: String ## Omit)
Let define following polymorphic function
object classifiers extends Poly1 {
implicit def stringClassifier[K <: Symbol](implicit witness: Witness.Aux[K]) =
at[FieldType[K, String]](value => Set(witness.value.name.toUpperCase))
implicit def intClassifier[K <: Symbol](implicit witness: Witness.Aux[K]) =
at[FieldType[K, Int]](value => {
val name = witness.value.name
Set(name.toUpperCase, name.toLowerCase)
})
implicit def omitClassifier[K, T] =
at[FieldType[K, T ## Omit]](_ => Set.empty[String])
}
Now your modelClassifier could be done as:
def modelClassifier: Classifier[Model] =
m => LabelledGeneric[Model].to(m).map(classifiers).toList.reduce(_ union _)
you can check it via
println(modelClassifier(Model(1, "b", "c", omit("d"))))
Note that Type ## Tag is subtype of Type so model.d still could be used as String everywhere
How do you intend to distinguish between bClassifier and cClassifier? By name? By order of declaration? That does not sound very "smart" or reliable. Consider encoding your intent explicitly instead. Something like this, perhaps:
case class Classifiable[T](data: T, classifier: Classifier[T])
object Classifiable {
def Null[T](data: T) = Classifiable(data, _ => Nil)
}
case class Model(a: Classifiable[Int], b: Classifiable[String], c: Classifiable[String], d: Classifiable[String])
object Model {
def apply(a: Int, b: String, c: String, d: String) =
Model(
Classifiable(a, aClassifier),
Classifiable(b, bClassifier),
Classifiable(c, cClassifier),
Classifiable.Null(d)
)
}
def modelClassifier(m: Model) = m
.productIterator
.collect { case x: Classifiable[_] =>
x.classifier()(x)
}
.reduce(_ ++ _)
println(modelClassifier(Model(1,"b", "c", "d")))

Scala: convert map to case class

Let's say I have this example case class
case class Test(key1: Int, key2: String, key3: String)
And I have a map
myMap = Map("k1" -> 1, "k2" -> "val2", "k3" -> "val3")
I need to convert this map to my case class in several places of the code, something like this:
myMap.asInstanceOf[Test]
What would be the easiest way of doing that? Can I somehow use implicit for this?
Two ways of doing this elegantly. The first is to use an unapply, the second to use an implicit class (2.10+) with a type class to do the conversion for you.
1) The unapply is the simplest and most straight forward way to write such a conversion. It does not do any "magic" and can readily be found if using an IDE. Do note, doing this sort of thing can clutter your companion object and cause your code to sprout dependencies in places you might not want:
object MyClass{
def unapply(values: Map[String,String]) = try{
Some(MyClass(values("key").toInteger, values("next").toFloat))
} catch{
case NonFatal(ex) => None
}
}
Which could be used like this:
val MyClass(myInstance) = myMap
be careful, as it would throw an exception if not matched completely.
2) Doing an implicit class with a type class creates more boilerplate for you but also allows a lot of room to expand the same pattern to apply to other case classes:
implicit class Map2Class(values: Map[String,String]){
def convert[A](implicit mapper: MapConvert[A]) = mapper conv (values)
}
trait MapConvert[A]{
def conv(values: Map[String,String]): A
}
and as an example you'd do something like this:
object MyObject{
implicit val new MapConvert[MyObject]{
def conv(values: Map[String, String]) = MyObject(values("key").toInt, values("foo").toFloat)
}
}
which could then be used just as you had described above:
val myInstance = myMap.convert[MyObject]
throwing an exception if no conversion could be made. Using this pattern converting between a Map[String, String] to any object would require just another implicit (and that implicit to be in scope.)
Here is an alternative non-boilerplate method that uses Scala reflection (Scala 2.10 and above) and doesn't require a separately compiled module:
import org.specs2.mutable.Specification
import scala.reflect._
import scala.reflect.runtime.universe._
case class Test(t: String, ot: Option[String])
package object ccFromMap {
def fromMap[T: TypeTag: ClassTag](m: Map[String,_]) = {
val rm = runtimeMirror(classTag[T].runtimeClass.getClassLoader)
val classTest = typeOf[T].typeSymbol.asClass
val classMirror = rm.reflectClass(classTest)
val constructor = typeOf[T].decl(termNames.CONSTRUCTOR).asMethod
val constructorMirror = classMirror.reflectConstructor(constructor)
val constructorArgs = constructor.paramLists.flatten.map( (param: Symbol) => {
val paramName = param.name.toString
if(param.typeSignature <:< typeOf[Option[Any]])
m.get(paramName)
else
m.get(paramName).getOrElse(throw new IllegalArgumentException("Map is missing required parameter named " + paramName))
})
constructorMirror(constructorArgs:_*).asInstanceOf[T]
}
}
class CaseClassFromMapSpec extends Specification {
"case class" should {
"be constructable from a Map" in {
import ccFromMap._
fromMap[Test](Map("t" -> "test", "ot" -> "test2")) === Test("test", Some("test2"))
fromMap[Test](Map("t" -> "test")) === Test("test", None)
}
}
}
Jonathan Chow implements a Scala macro (designed for Scala 2.11) that generalizes this behavior and eliminates the boilerplate.
http://blog.echo.sh/post/65955606729/exploring-scala-macros-map-to-case-class-conversion
import scala.reflect.macros.Context
trait Mappable[T] {
def toMap(t: T): Map[String, Any]
def fromMap(map: Map[String, Any]): T
}
object Mappable {
implicit def materializeMappable[T]: Mappable[T] = macro materializeMappableImpl[T]
def materializeMappableImpl[T: c.WeakTypeTag](c: Context): c.Expr[Mappable[T]] = {
import c.universe._
val tpe = weakTypeOf[T]
val companion = tpe.typeSymbol.companionSymbol
val fields = tpe.declarations.collectFirst {
case m: MethodSymbol if m.isPrimaryConstructor ⇒ m
}.get.paramss.head
val (toMapParams, fromMapParams) = fields.map { field ⇒
val name = field.name
val decoded = name.decoded
val returnType = tpe.declaration(name).typeSignature
(q"$decoded → t.$name", q"map($decoded).asInstanceOf[$returnType]")
}.unzip
c.Expr[Mappable[T]] { q"""
new Mappable[$tpe] {
def toMap(t: $tpe): Map[String, Any] = Map(..$toMapParams)
def fromMap(map: Map[String, Any]): $tpe = $companion(..$fromMapParams)
}
""" }
}
}
This works well for me,if you use jackson for scala:
def from[T](map: Map[String, Any])(implicit m: Manifest[T]): T = {
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.convertValue(map)
}
Reference from:Convert a Map<String, String> to a POJO
I don't love this code, but I suppose this is possible if you can get the map values into a tuple and then use the tupled constructor for your case class. That would look something like this:
val myMap = Map("k1" -> 1, "k2" -> "val2", "k3" -> "val3")
val params = Some(myMap.map(_._2).toList).flatMap{
case List(a:Int,b:String,c:String) => Some((a,b,c))
case other => None
}
val myCaseClass = params.map(Test.tupled(_))
println(myCaseClass)
You have to be careful to make sure the list of values is exactly 3 elements and that they are the correct types. If not, you end up with a None instead. Like I said, not great, but it shows that it is possible.
commons.mapper.Mappers.mapToBean[CaseClassBean](map)
Details: https://github.com/hank-whu/common4s
Here's an update to Jonathon's answer for Scala 3 (which no longer has TypeTag). Be aware that this won't work for case classes nested inside of other classes. But for top-level case classes it seems to work fine.
import scala.reflect.ClassTag
object Reflect:
def fromMap[T <: Product : ClassTag](m: Map[String, ?]): T =
val classTag = implicitly[ClassTag[T]]
val constructor = classTag.runtimeClass.getDeclaredConstructors.head
val constructorArgs = constructor.getParameters()
.map { param =>
val paramName = param.getName
if (param.getType == classOf[Option[_]])
m.get(paramName)
else
m.get(paramName)
.getOrElse(throw new IllegalArgumentException(s"Missing required parameter: $paramName"))
}
constructor.newInstance(constructorArgs: _*).asInstanceOf[T]
And a test for the above:
case class Foo(a: String, b: Int, c: Option[String] = None)
case class Bar(a: String, b: Int, c: Option[Foo])
class ReflectSuite extends munit.FunSuite:
test("fromMap") {
val m = Map("a" -> "hello", "b" -> 42, "c" -> "world")
val foo = Reflect.fromMap[Foo](m)
assertEquals(foo, Foo("hello", 42, Some("world")))
val n = Map("a" -> "hello", "b" -> 43)
val foo2 = Reflect.fromMap[Foo](n)
assertEquals(foo2, Foo("hello", 43))
val o = Map("a" -> "yo", "b" -> 44, "c" -> foo)
val bar = Reflect.fromMap[Bar](o)
assertEquals(bar, Bar("yo", 44, Some(foo)))
}
test("fromMap should fail when required parameter is missing") {
val m = Map("a" -> "hello", "c" -> "world")
intercept[java.lang.IllegalArgumentException] {
Reflect.fromMap[Foo](m)
}
}

Serialize a map that doesn't have a string as a key with lift-json

It seems like lift-json is limited to maps that have Strings as keys.
What is the best way to bypass this limitation ?
Define your own Serializer[Map[Any, Any]].
import net.liftweb.json._
import ext._
object MapSerializer extends Serializer[Map[Any, Any]] {
def serialize(implicit format: Formats): PartialFunction[Any, JValue] = {
case m: Map[_, _] => JObject(m.map({
case (k, v) => JField(
k match {
case ks: String => ks
case ks: Symbol => ks.name
case ks: Any => ks.toString
},
Extraction.decompose(v)
)
}).toList)
}
def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), Map[Any, Any]] = {
sys.error("Not interested.")
}
}
Then add it to the implicit Formats variable.
implicit val formats = DefaultFormats + MapSerializer
That's all.
In addition to the previous answer you can define instead:
def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), Map[Any, Any]] = {
Map()
}
This doesn't break any other working map deserialization.