I have a schema which is defined like this
#JsonDeserialize(using = classOf[ProductDeserializer])
trait GenericProduct{
#JsonUnwrapped
def commonAttributes: CommonAttributes
}
There are different kinds of products. For example, Product A given below extends base trait Product. Similarly, there are other products B, C, D.
case class ProductA(commonAttributes: CommonAttributes,
extraField1: String,
extraField2: String) extends GenericProduct
My serializer method is like this
#Component
class ProductSerializer #Autowired()
(val mapper: ScalaObjectMapper = ObjectMapperSingleton.getMapper) {
def serialize[T<:GenericProduct](product: T)(implicit m: Manifest[T]): String = {
mapper
.writerWithType[T]
.writeValueAsString(product)
}
}
If i pass objects of type A, B, C, D, it works just fine. Serialized JSON string has common attributes and product specific attributes.
But if i wrap the product objects inside a case class. For example, from one method i'm returning something like this
case class ConsolidatedProducts(newProducts: List[GenericProduct],
oldProducts: List[GenericProduct])
It doesn't complain during compile time but during runtime the output have only the common attributes.
Question: Is this because of type erasure?
If I pass the manifest implicitly to all the places where the consolidatedProducts is called and used, then i can see the expected JSON with all the attributes which makes me believe this is because of type erasure.
Found two ways to solve this
Use manifest implicitly everywhere to retain type information
Just update writerWithType[T] to writerWithType[T.type]
Is there any other workaround? I'm still not sure why the second solution works. If someone can explain, that would be great.
Related
Abstract problem: Create a trait that can be mixed into the companion object of a class, to give that object a method that returns an object of that class.
Concrete problem: I'm trying to create a bunch of classes for use with RESTful service calls, that know how to serialize and de-serialize themselves, like so:
case class Foo
(
var bar : String,
var blip : String
)
extends SerializeToJson
object Foo extends DeserializeFromJson
The intended usage is like so:
var f = Foo( "abc","123" )
var json = f.json
var newF = Foo.fromJson( json )
I'm using Genson to do the serialization/deserialization, which I access through a global object:
object JSON {
val parser = new ScalaGenson( new GensonBuilder() <...> )
}
Then I define the traits like so:
trait SerializeToJson {
def json : String = JSON.parser.toJson(this)
}
trait DeserializeFromJson[T <: DeserializeFromJson[T]] {
def fromJson( json : String ) : T = JSON.parser.fromJson( json )
}
This compiles. But this does not:
object Foo extends DeserializeFromJson[Foo]
I get the following error message:
type arguments [Foo] do not conform to trait DeserializeFromJson's
type parameter bounds [T <: DeserializeFromJson[T]]
I've tried creating a single trait, like so:
trait JsonSerialization[T <: JsonSerialization[T]] {
def json(implicit m: Manifest[JsonSerialization[T]]) : String =
JSON.parser.toJson(this)(m)
def fromJson( json : String ) : T =
JSON.parser.fromJson(json)
}
Now, if I just declare case class Foo (...) extends JsonSerialization[Foo] then I can't call Foo.fromJson because only an instance of class Foo has that method, not the companion object.
If I declare object Foo extend JsonSerialization[Foo] then I can compile and Foo has a .fromJson method. But at run time, the call to fromJson thinks that T is a JsonSerialization, and not a Foo, or so the following run-time error suggests:
java.lang.ClassCastException: scala.collection.immutable.HashMap$HashTrieMap cannot be cast to ...JsonSerialization
at ...JsonSerialization$class.fromJson(DataModel.scala:14)
at ...Foo.fromJson(Foo.scala:6)
And I can't declare object Foo extends Foo because I get
module extending its companion class cannot use default constructor arguments
So I can try adding constructor parameters, and that compiles and runs, but again the run-time type when it tries to deserialize is wrong, giving me the above error.
The only thing I've been able to do that works is to define fromJson in every companion object. But there MUST be a way to define it in a trait, and just mix in that trait. Right?
The solution is to simplify the type parameter for the trait.
trait DeserializeFromJson[T] {
def fromJson( json : String )(implicit m : Manifest[T]) : T =
JSON.parser.fromJson[T](json)(m)
}
Now, the companion object can extend DeserializeFromJson[Foo] and when I call Foo.fromJson( json ) it is able to tell Genson the correct type information so that an object of the appropriate type is created.
The problem is related to how implicits work.
Genson expects a Manifest that it will use to know to what type it must deserialize. This manifest is defined as implicit in Genson, meaning that it will try to get it from implicitly available manifests in the "caller code". However in your original version there is no Manifest[T] in DeserializeFromJson.
An alternate way would be to define the DeserializeFromJson like that (which will just produce a constructor with an implicit Manifest[T] argument):
abstract class DeserializeFromJson[T: Manifest] {
def fromJson( json : String ) : T = JSON.parser.fromJson[T](json)
}
object Foo extends DeserializeFromJson[Foo]
More generally if you don't bring more value by encapsulating a lib (in this case Genson), I think you shouldn't do that. As you basically reduce the features of Genson (now people can only work with strings) and introduce problems like the one you hit.
I think your type parameter constraint were originally wrong;
you had
trait DeserializeFromJson[T <: DeserializeFromJson[T]]
With your own answer, you fully relaxed it; you needed
trait DeserializeFromJson[T <: SerializeToJson]
...which the error was trying to tell you.
The need for the implicit Manifest (ClassTag now I believe) or context-bounds was on the money.
Would be nice for Scala to allow the specification of inheritance and type-parameter constraints based on class/trait and companion object relationship, given it is already aware, to some degree, when it comes to access-modifiers and implicit scopes.
Please note I am learning Scala so what I suggest may not be the best (idomatic) way of achieving this, therefore I'll describe the problem I'm trying to solve first, then my current implementation!
Problem: Given some input document, e.g. xml or json, create an object Doc with its raw contents as a variable, apply a sequence of FieldExtractors which extract a number of value, i.e. Fields, which are stored on the Doc object and can be accessed in a type safe manner later, e.g. val username: String = doc.getField(UsernameField)
N.B everything must be Serializable so it can be passed over the wire via a particular framework
So, my current attempt:
abstract class Field[+T <: Serializable](val name: String, val valueType: Class[T])
trait Fields {
var fields: mutable.HashMap[Class[Field[Serializable]], Field[Serializable]] = mutable.HashMap()
def hasField[T <: Serializable](field: Field[T]): Boolean = false
def getField[T <: Serializable](field: Field[T]): T = fields.get(field).asInstanceOf(T)
def setField[T <: Serializable](field: Field[T], value: T): Unit = fields.put(field, value)
}
class Doc(val rawData: String) with Fields
abstract class FieldExtractor[+TYPE <: Serializable](val field: Field[TYPE]) {
def extractField(input: Doc): Option[TYPE]
}
But I get all sorts of errors such as:
Error:(14, 39) inferred type arguments [String] do not conform to method hasField's type parameter bounds [T <: Serializable]
val result:Boolean = fieldsObject.hasField(field)
^
I'm wondering if maybe I should use Value types instead? Or the https://github.com/mikaelv/strucs project (which seems rather young)? Or whether there's a better approach?
Eventually I'd write something like
extractors.foldLeft(doc)((doc, extractor) => doc.setField(field, extractor.extractField(doc); doc
This appears to be a symptom of Scala having its own Serializable class.
String (which Scala uses as a synonym for java.lang.String) implements Java's java.io.Serializable whereas your Field class has declared the generic bounds using Scala's Serializable class.
If you change the declaration to this:
abstract class Field[+T <: java.io.Serializable](val name: String, val valueType: Class[T])
it compiles* and does the type safe get/set checks correctly for me (* using Scala SDK 2.11.7, I had to correct some minor typos).
Note, there's also an error in the Fields class. I think you meant to declare the hash map as:
fields: mutable.HashMap[Field[Serializable], Serializable]
This is a follow up to the following question, which concerned serialization: How best to keep a cached list of member fields, one each for a family of case classes in Scala
I'm trying to generically support deserialization in the same way. One straightforward attempt is the following:
abstract class Serializer[T](implicit ctag: ClassTag[T]) {
private val fields = ctag.runtimeClass.getDeclaredFields.toList
fields foreach { _.setAccessible(true) }
implicit class AddSerializeMethod(obj: T) {
def serialize = fields.map(f => (f.getName, f.get(obj)))
}
def deserialize(data: List[(String, Any)]): T = {
val m = data toMap
val r: T = ctag.runtimeClass.newInstance // ???
fields.foreach { case f => f.set(r, m(f.getName)) }
r;
}
}
There are a couple of issues with the code:
The line with val r: T = ... has a compile error because the compiler thinks it's not guaranteed to have the right type. (I'm generally unsure of how to create a new instance of a generic class in a typesafe way -- not sure why this isn't safe since the instance of Serializer is created with a class tag whose type is checked by the compiler).
The objects I'm creating are expected to be immutable case class objects, which are guaranteed to be fully constructed if created in the usual way. However, since I'm mutating the fields of instances of these objects in the deserialize method, how can I be sure that the objects will not be seen as partially constructed (due to caching and instruction reordering) if they are published to other threads?
ClassTag's runtimeClass method returns Class[_], not Class[T], probably due to the fact generics in Scala and Java behave differently; you can try casting it forcefully: val r: T = ctag.runtimeClass.newInstance.asInstanceOf[T]
newInstance calls the default, parameterless constructor. If the class doesn't have one, newInstance will throw InstantiationException. There's no way around it, except for:
looking around for other constructors
writing custom serializers (see how Gson does that; BTW Gson can automatically serialize only classes with parameterless constructors and those classes it has predefined deserializers for)
for case classes, finding their companion object and calling its apply method
Anyhow, reflection allows for modifying final fields as well, so if you manage to create an immutable object, you'll be able to set its fields.
I'm having trouble finding an elegant way of designing a some simple classes to represent HTTP messages in Scala.
Say I have something like this:
abstract class HttpMessage(headers: List[String]) {
def addHeader(header: String) = ???
}
class HttpRequest(path: String, headers: List[String])
extends HttpMessage(headers)
new HttpRequest("/", List("foo")).addHeader("bar")
How can I make the addHeader method return a copy of itself with the new header added? (and keep the current value of path as well)
Thanks,
Rob.
It is annoying but the solution to implement your required pattern is not trivial.
The first point to notice is that if you want to preserve your subclass type, you need to add a type parameter. Without this, you are not able to specify an unknown return type in HttpMessage
abstract class HttpMessage(headers: List[String]) {
type X <: HttpMessage
def addHeader(header: String):X
}
Then you can implement the method in your concrete subclasses where you will have to specify the value of X:
class HttpRequest(path: String, headers: List[String])
extends HttpMessage(headers){
type X = HttpRequest
def addHeader(header: String):HttpRequest = new HttpRequest(path, headers :+header)
}
A better, more scalable solution is to use implicit for the purpose.
trait HeaderAdder[T<:HttpMessage]{
def addHeader(httpMessage:T, header:String):T
}
and now you can define your method on the HttpMessage class like the following:
abstract class HttpMessage(headers: List[String]) {
type X <: HttpMessage
def addHeader(header: String)(implicit headerAdder:HeaderAdder[X]):X = headerAdder.add(this,header) }
}
This latest approach is based on the typeclass concept and scales much better than inheritance. The idea is that you are not forced to have a valid HeaderAdder[T] for every T in your hierarchy, and if you try to call the method on a class for which no implicit is available in scope, you will get a compile time error.
This is great, because it prevents you to have to implement addHeader = sys.error("This is not supported")
for certain classes in the hierarchy when it becomes "dirty" or to refactor it to avoid it becomes "dirty".
The best way to manage implicit is to put them in a trait like the following:
trait HeaderAdders {
implicit val httpRequestHeaderAdder:HeaderAdder[HttpRequest] = new HeaderAdder[HttpRequest] { ... }
implicit val httpRequestHeaderAdder:HeaderAdder[HttpWhat] = new HeaderAdder[HttpWhat] { ... }
}
and then you provide also an object, in case user can't mix it (for example if you have frameworks that investigate through reflection properties of the object, you don't want extra properties to be added to your current instance) (http://www.artima.com/scalazine/articles/selfless_trait_pattern.html)
object HeaderAdders extends HeaderAdders
So for example you can write things such as
// mixing example
class MyTest extends HeaderAdders // who cares about having two extra value in the object
// import example
import HeaderAdders._
class MyDomainClass // implicits are in scope, but not mixed inside MyDomainClass, so reflection from Hiberante will still work correctly
By the way, this design problem is the same of Scala collections, with the only difference that your HttpMessage is TraversableLike. Have a look to this question Calling map on a parallel collection via a reference to an ancestor type
There seems to be a lot of enthusiasm among Scala bloggers lately for the type classes pattern, in which a simple class has functionality added to it by an additional class conforming to some trait or pattern. As a vastly oversimplified example, the simple class:
case class Wotsit (value: Int)
can be adapted to the Foo trait:
trait Foo[T] {
def write (t: T): Unit
}
with the help of this type class:
implicit object WotsitIsFoo extends Foo[Wotsit] {
def write (wotsit: Wotsit) = println(wotsit.value)
}
The type class is typically captured at compile time with implicts, allowing both the Wotsit and its type class to be passed together into a higher order function:
def writeAll[T] (items: List[T])(implicit tc: Foo[T]) =
items.foreach(w => tc.write(w))
writeAll(wotsits)
(before you correct me, I said it was an oversimplified example)
However, the use of implicits assumes that the precise type of the items is known at compile time. I find in my code this often isn't the case: I will have a list of some type of item List[T], and need to discover the correct type class to work on them.
The suggested approach of Scala would appear to be to add the typeclass argument at all points in the call hierarchy. This can get annoying as an the code scales and these dependencies need to be passed down increasingly long chains, through methods to which they are increasingly irrelevant. This makes the code cluttered and harder to maintain, the opposite of what Scala is for.
Typically this is where dependency injection would step in, using a library to supply the desired object at the point it's needed. Details vary with the library chosen for DI - I've written my own in Java in the past - but typically the point of injection needs to define precisely the object desired.
Trouble is, in the case of a type class the precise value isn't known at compile time. It must be selected based on a polymorphic description. And crucially, the type information has been erased by the compiler. Manifests are Scala's solution to type erasure, but it's far from clear to me how to use them to address this issue.
What techniques and dependency injection libraries for Scala would people suggest as a way of tackling this? Am I missing a trick? The perfect DI library? Or is this really the sticking point it seems?
Clarification
I think there are really two aspects to this. In the first case, the point where the type class is needed is reached by direct function calls from the point where the exact type of its operand is known, and so sufficient type wrangling and syntactic sugar can allow the type class to be passed to the point it's needed.
In the second case, the two points are separated by a barrier - such as an API that can't be altered, or being stored in a database or object store, or serialised and send to another computer - that means the type class can't be passed along with its operand. In this case, given an object whose type and value are known only at runtime, the type class needs somehow to be discovered.
I think functional programmers have a habit of assuming the first case - that with a sufficiently advanced language, the type of the operand will always be knowable. David and mkniessl provided good answers for this, and I certainly don't want to criticise those. But the second case definitely does exist, and that's why I brought dependency injection into the question.
A fair amount of the tediousness of passing down those implicit dependencies can be alleviated by using the new context bound syntax. Your example becomes
def writeAll[T:Foo] (items: List[T]) =
items.foreach(w => implicitly[Foo[T]].write(w))
which compiles identically but makes for nice and clear signatures and has fewer "noise" variables floating around.
Not a great answer, but the alternatives probably involve reflection, and I don't know of any library that will just make this automatically work.
(I have substituted the names in the question, they did not help me think about the problem)
I'll attack the problem in two steps. First I show how nested scopes avoid having to declare the type class parameter all the way down its usage. Then I'll show a variant, where the type class instance is "dependency injected".
Type class instance as class parameter
To avoid having to declare the type class instance as implicit parameter in all intermediate calls, you can declare the type class instance in a class defining a scope where the specific type class instance should be available. I'm using the shortcut syntax ("context bound") for the definition of the class parameter.
object TypeClassDI1 {
// The type class
trait ATypeClass[T] {
def typeClassMethod(t: T): Unit
}
// Some data type
case class Something (value: Int)
// The type class instance as implicit
implicit object SomethingInstance extends ATypeClass[Something] {
def typeClassMethod(s: Something): Unit =
println("SomthingInstance " + s.value)
}
// A method directly using the type class
def writeAll[T:ATypeClass](items: List[T]) =
items.foreach(w => implicitly[ATypeClass[T]].typeClassMethod(w))
// A class defining a scope with a type class instance known to be available
class ATypeClassUser[T:ATypeClass] {
// bar only indirectly uses the type class via writeAll
// and does not declare an implicit parameter for it.
def bar(items: List[T]) {
// (here the evidence class parameter defined
// with the context bound is used for writeAll)
writeAll(items)
}
}
def main(args: Array[String]) {
val aTypeClassUser = new ATypeClassUser[Something]
aTypeClassUser.bar(List(Something(42), Something(4711)))
}
}
Type class instance as writable field (setter injection)
A variant of the above which would be usable using setter injection. This time the type class instance is passed via a setter call to the bean using the type class.
object TypeClassDI2 {
// The type class
trait ATypeClass[T] {
def typeClassMethod(t: T): Unit
}
// Some data type
case class Something (value: Int)
// The type class instance (not implicit here)
object SomethingInstance extends ATypeClass[Something] {
def typeClassMethod(s: Something): Unit =
println("SomthingInstance " + s.value)
}
// A method directly using the type class
def writeAll[T:ATypeClass](items: List[T]) =
items.foreach(w => implicitly[ATypeClass[T]].typeClassMethod(w))
// A "service bean" class defining a scope with a type class instance.
// Setter based injection style for simplicity.
class ATypeClassBean[T] {
implicit var aTypeClassInstance: ATypeClass[T] = _
// bar only indirectly uses the type class via writeAll
// and does not declare an implicit parameter for it.
def bar(items: List[T]) {
// (here the implicit var is used for writeAll)
writeAll(items)
}
}
def main(args: Array[String]) {
val aTypeClassBean = new ATypeClassBean[Something]()
// "inject" the type class instance
aTypeClassBean.aTypeClassInstance = SomethingInstance
aTypeClassBean.bar(List(Something(42), Something(4711)))
}
}
Note that the second solution has the common flaw of setter based injection that you can forget to set the dependency and get a nice NullPointerException upon use...
The argument against type classes as dependency injection here is that with type classes the "precise type of the items is known at compile time" whereas with dependency injection, they are not. You might be interested in this Scala project rewrite effort where I moved from the cake pattern to type classes for dependency injection. Take a look at this file where the implicit declarations are made. Notice how the use of environment variables determines the precise type? That is how you can reconcile the compile time requirements of type classes with the run time needs of dependency injection.