json4s Serialize and deserialize generic type - scala

Because I am dealing with generic type therefore I can't use specific case classes. Then I created a generic util which serializes and deserializes generic object.
import org.json4s
import org.json4s.Formats._
import org.json4s.native.JsonMethods._
object JsonHelper {
def json2Object[O](input: String) : O = {
parse(json4s.string2JsonInput(input)).asInstanceOf[O]
}
def object2Json[O](input: O) : String = {
write(input).toString
}
}
The compiler throws the error:
No JSON serializer found for type O. Try to implement an implicit Writer or JsonFormat for this type.
write(input).toString
This should be thrown at runtime but why it's thrown at compile time?

In a comment above, you asked "So how jackson can work with java object? It use reflection right? And why it's different from Scala?", which gets to the heart of this question.
The json4s "native" serializer you have imported uses compile-time reflection to create the Writer.
Jackson uses run-time reflection to do the same.
The compile-time version is more efficient; the run-time version is more flexible.
To use the compile-time version, you need to let the compiler have enough information to choose the correct Writer based on the declared type of the object to be serialized. This will rule out very generic writer methods like the one you propose. See #TimP's answer for how to fix your code for that version.
To use the run-time version, you can use Jackson via the org.json4s.jackson.JsonMethods._ package. See https://github.com/json4s/json4s#jackson

The compiler error you posted comes from this location in the json4s code. The write function you're calling takes an implicit JSON Writer, which is how the method can take arbitrary types. It's caught at compile time because implicit arguments are compiled the same way explicit ones are -- it's as if you had:
def f(a: Int, b: Int) = a + b
f(5) // passed the wrong number of arguments
I'm having a bit of trouble seeing exactly which write method you're calling here -- the json4s library is pretty big and things are overloaded. Can you paste the declared write method you're using? It almost certainly has a signature like this:
def write[T](value: T)(implicit writer: Writer[T]): JValue
If it looks like the above, try including the implicit writer parameter in your method as so:
object JsonHelper {
def json2Object[O](input: String)(implicit reader: Reader[O]) : O = {
parse(json4s.string2JsonInput(input)).asInstanceOf[O]
}
def object2Json[O](input: O)(implicit writer: Writer[O]) : String = {
write(input).toString
}
}

In this example, you have a deal with generic types, Scala, like another jvm languages, has type erasing mechanism at compile time (error message at compile time may don't contain message about generic in a whole), so try to append this fragment to the signature of both methods:
(implicit tag: ClassTag[T])
it's similar to you example with generic, but with jackson.
HTH

Related

How to mock scala generic method

I have a traits(below simple versions) and I want to mock Foo.
trait MyTrait[A]
trait Foo {
def bar[T: MyTrait](id:Int, data: T, other:Option[String] = None): String
}
I tried:
implicit val myTrait = new MyTrait[String] {}
val src = mock[Foo]
when(src.bar(any(),any(),any())).thenReturn("ok")
src.bar(1, "some", None)
It fails with:
Invalid use of argument matchers! 4 matchers expected, 3 recorded
How to mock this kind of method?
Mockito does not play well with scala code. What happens in your case is that
notify.bar(any(), any(), any()) has to be invoked inside when(), but type of the bar method is unknown, so when scalac is looking-up implicits there are possibly several instances of Writes that fit here (becuase all of them do).
You can do something like this to make it work:
when(src.bar[T](any(),any(),any())).thenReturn("ok")
Edit
Following your edit, i think you should reconsider usage of mockito in the first place. Here's what happens:
Foo has the following signature after desugaring
trait Foo {
def bar[T](id:Int, data: T, other:Option[String] = None)(implicit ev: MyTrait[T]): String
}
I don't know if you are aware how does mockito work, but here is a quick explanation why (from what i can tell) this error happens:
at runtime method bar has the following "signature" (due to type erasure):
bar(id: Int, data: Object, other: Option[Object], ev: MyTrait[Object])
when(src.bar[T](any(),any(),any())).thenReturn("ok") actually invokes this method on a proxy object and registers "matchers", but ev is passed myTrait instead of a matcher, so i guess this violates some constraints of the library
As side-note: usually mocking is not that hard and you can simply implement a "mocked" trait without any help from mockito or other similar library.
The syntax [T : MyTrait] is sugar to add (implicit typeClass: MyTrait[T]) as a second set of arguments, so basically you're missing an argument matcher as the error states
So if you do
val src = mock[Foo]
when(src.bar(any(),any(),any())(any())).thenReturn("ok")
it works as expected
BTW, this is a bit of self promotion but I just published a library called mockito-scala that improves the mockito syntax for Scala, is part of the mockito ecosystem so hopefully should become the default when working with Scala, you can find it here https://github.com/mockito/mockito-scala with the information to get the dependency and what problems does it actually solves.

Scala function return type based on generic

Using Scala generics I'm trying to abstract some common functions in my Play application. The functions return Seqs with objects deserialized from a REST JSON service.
def getPeople(cityName: String): Future[Seq[People]] = {
getByEndpoint[People](s"http://localhost/person/$cityName")
}
def getPeople(): Future[Seq[Dog]] = {
getByEndpoint[Dog]("http://localhost/doge")
}
The fetch and deserialization logic is packed into a single function using generics.
private def getByEndpoint[T](endpoint: String): Future[Seq[T]] = {
ws.url(endpoint)
.get()
.map(rsp => rsp.json)
.flatMap { json =>
json.validate[Seq[T]] match {
case s: JsSuccess[Seq[T]] =>
Future.successful(s.get)
case e: JsError =>
Future.failed(new RuntimeException(s"Get by endpoint JSON match failed: $e"))
}
}
}
Problem is is I'm getting "No Json deserializer found for type Seq[T]. Try to implement an implicit Reads or Format for this type.". I'm sure I'm not using T properly in Seq[T] (according to my C#/Java memories at least), but I can't find any clue how to do it the proper way in Scala. Everything works as expected without using generics.
Play JSON uses type classes to capture information about which types can be (de-)serialized to and from JSON, and how. If you have an implicit value of type Format[Foo] in scope, that's referred to as an instance of the Format type class for Foo.
The advantage of this approach is that it gives us a way to constrain generic types (and have those constraints checked at compile time) that doesn't depend on subtyping. For example, there's no way the standard library's String will ever extend some kind of Jsonable trait that Play (or any other library) might provide, so we need some way of saying "we know how to encode Strings as JSON" that doesn't involve making String a subtype of some trait we've defined ourselves.
In Play JSON you can do this by defining implicit Format instances, and Play itself provides many of these for you (e.g., if you've got one for T, it'll give you one for Seq[T]). The validate method on JsValue requires one of these instances (actually a subtype of Format, Reads, but that's not terribly relevant here) for its type parameter—Seq[T] in this case—and it won't compile unless the compiler can find that instance.
You can provide this instance by adding the constraint to your own generic method:
private def getByEndpoint[T: Format](endpoint: String): Future[Seq[T]] = {
...
}
Now with the T: Format syntax you've specified that there has to be a Format instance for T (even though you don't constraint T in any other way), so the compiler knows how to provide the Format instance for Seq[T] that the json.validate[Seq[T]] call requires.

Possible to find parameter type methods return type in Scala where parameter is a primitive type?

Suppose I have:
class X
{
val listPrimitive: List[Int] = null
val listX: List[X] = null
}
and I print out the return types of each method in Scala as follows:
classOf[ComplexType].getMethods().foreach { m => println(s"${m.getName}: ${m.getGenericReturnType()}") }
listPrimitive: scala.collection.immutable.List<Object>
listX: scala.collection.immutable.List<X>
So... I can determine that the listX's element type is X, but is there any way to determine via reflection that listPrimitive's element type is actually java.lang.Integer? ...
val list:List[Int] = List[Int](123);
val listErased:List[_] = list;
println(s"${listErased(0).getClass()}") // java.lang.Integer
NB. This seems not to be an issue due to JVM type erasure since I can find the types parameter of List. It looks like the scala compiler throws away this type information IFF the parameter type is java.lang.[numbers] .
UPDATE:
I suspect this type information is available, due to the following experiment. Suppose I define:
class TestX{
def f(x:X):Unit = {
val floats:List[Float] = x.listPrimitive() // type mismatch error
}
}
and X.class is imported via a jar. The full type information must be available in X.class in order that this case correctly fails to compile.
UPDATE2:
Imagine you're writing a scala extension to a Java serialization library. You need to implement a:
def getSerializer(clz:Class[_]):Serializer
function that needs to do different things depending on whether:
clz==List[Int] (or equivalently: List[java.lang.Integer])
clz==List[Float] (or equivalently: List[java.lang.Float])
clz==List[MyClass]
My problem is that I will only ever see:
clz==List[Object]
clz==List[Object]
clz==List[MyClass]
because clz is provided to this function as clz.getMethods()(i).getGenericReturnType().
Starting with clz:Class[_] how can I recover the element type information that was lost?
Its not clear to me that TypeToken will help me because its usages:
typeTag[T]
requires that I provide T (ie. at compile time).
So, one path to a solution... Given some clz:Class[_], can I determine the TypeTokens of its method's return types? Clearly this is possible as this information must be contained (somewhere) in a .class file for a scala compiler to correctly generate type mismatch errors (see above).
At the java bytecode level Ints have to be represented as something else (apparently Object) because a List can only contain objects, not primitives. So that's what java-level reflection can tell you. But the scala type information is, as you infer, present (at the bytecode level it's in an annotation, IIRC), so you should be able to inspect it with scala reflection:
import scala.reflect.runtime.universe._
val list:List[Int] = List[Int](123)
def printTypeOf[A: TypeTag](a: A) = println(typeOf[A])
printTypeOf(list)
Response to update2: you should use scala reflection to obtain a mirror, not the Class[_] object. You can go via the class name if need be:
import scala.reflect.runtime.universe._
val rm = runtimeMirror(getClass.getClassLoader)
val someClass: Class[_] = ...
val scalaMirrorOfClass = rm.staticClass(someClass.getName)
// or possibly rm.reflectClass(someClass) ?
val someObject: Any = ...
val scalaMirrorOfObject = rm.reflectClass(someObject)
I guess if you really only have the class, you could create a classloader that only loads that class? I can't imagine a use case where you wouldn't have the class, or even a value, though.

Can structural typing work with generics?

I have an interface defined using a structural type like this:
trait Foo {
def collection: {
def apply(a: Int) : String
def values() : collection.Iterable[String]
}
}
}
I wanted to have one of the implementers of this interface do so using a standard mutable HashMap:
class Bar {
val collection: HashMap[Int, String] = HashMap[Int, String]()
}
It compiles, but at runtime I get a NoSuchMethod exception when referring a Bar instance through a Foo typed variable. Dumping out the object's methods via reflection I see that the HashMap's apply method takes an Object due to type erasure, and there's some crazily renamed generated apply method that does take an int. Is there a way to make generics work with structural types? Note in this particular case I was able to solve my problem using an actual trait instead of a structural type and that is overall much cleaner.
Short answer is that the apply method parameter is causing you grief because it requires some implicit conversions of the parameter (Int => Integer). Implicits are resolved at compile time, the NoSuchMethodException is likely a result of these missing implicits.
Attempt to use the values method and it should work since there are no implicits being used.
I've attempted to find a way to make this example work but have had no success so far.

How to infer types when one is a return value?

I'm trying to define a method which is generic both in its parameter and return type. Basically to make a helper function for JSON serialization to/from case classes.
so I want to write something like this pseudocode:
def post[Request,Response](data:Request) : Response = ???
case class A(i:String)
case class B(j:Int)
val result = post[A,B]("input")
in this case (assuming no errors) result is of type B.
It's understandable that the compiler can't infer the return value, but I'd like it to infer the Request type. In other words I'd like to write something like
val result = post[B]("input")
where the type of A is inferred by the data parameter, so the user need only specify the return type when calling the function.
I don't know many details of Scala specifically, but in Haskell that ability is enabled by a compiler option called "Functional dependencies", whereby you have a typeclass with two type variables, one of which can be derived from the other - see section 7.4.3 of http://www.haskell.org/ghc/docs/6.6/html/users_guide/type-extensions.html. Obviously you can't just use this feature, since it's in a different language, but knowing what it's called should help you find a solution. For example, Functional dependencies in Scala looks like a good guess; although again, I don't know enough Scala to read that article and then tell you exactly how to answer your original JSON question.
Following on from #amalloy's answer, and the link he provides, the Scala equivalent for what you are trying to achieve would be something like the following:
trait ReqResp[Request,Response] {
def apply(req: Request): Response
}
def post[Request,Response](data:Request)(implicit rr: ReqResp[Request,Response]): Response = rr(data)
case class A(i:String)
case class B(j:Int)
implicit object reqRespAB extends ReqResp[A,B] {
def apply(a: A) = B(a.i.toInt)
}
val result = post(A("456"))
This gives the output:
result: B = B(456)