So I have an object like this
case class QueryResult(events: List[Event])
that I need to serialize deserialize. For serialization I have this implicit class
implicit class Serializer(obj: Object) {
def serialize: Array[Byte] = {
val stream = new ByteArrayOutputStream()
val objectOutputStream = new ObjectOutputStream(stream)
objectOutputStream.writeObject(obj)
objectOutputStream.close
stream.toByteArray
}
}
It's used like
val byteArray = QueryResult(events).serialize
Then for desrialization I have this trait that can be extended by the companion class.
trait Deserializer[A] {
private type resultType = A
def deserialize(bytes: Array[Byte]): A = {
val objectInputStream = new ObjectInputStream(
new ByteArrayInputStream(bytes)
)
val value = objectInputStream.readObject.asInstanceOf[resultType]
objectInputStream.close
value
}
}
You would use it like so
object QueryResult extends Deserializer[QueryResult]
Then you should be able to reconstruct an instance of QueryResult from a byte array like so
val QueryResult: QueryResult = QueryResult.deserialize(byteArray)
The issue is this sometimes causes an error like the one below
java.lang.ClassCastException: cannot assign instance of scala.collection.generic.DefaultSerializationProxy to field co.app.QueryResult.events of type scala.collection.immutable.List in instance of co.app.QueryResult
at java.base/java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2205)
at java.base/java.io.ObjectStreamClass$FieldReflector.checkObjectFieldValueTypes(ObjectStreamClass.java:2168)
at java.base/java.io.ObjectStreamClass.checkObjFieldValueTypes(ObjectStreamClass.java:1422)
at java.base/java.io.ObjectInputStream.defaultCheckFieldValues(ObjectInputStream.java:2450)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2357)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2166)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:482)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:440)
at co.app.Deserializer.deserialize(Types.scala:51)
at co.app.Deserializer.deserialize$(Types.scala:47)
at co.app.LocalStorage$QueryResults$.get(LocalStorage.scala:241)
... 36 elided
Now when I say sometimes I mean the deserialization works when I assemble the code into a jar and run it using the java cli, but it breaks when I try to execute that code via sbt test, sbt console, or ammonite. Any idea why it breaks under these circumstances and possible suggestions to fix?
Related
I am trying to do a very basic serialization of a very simple case class in Scala:
import org.scalatest.wordspec.AnyWordSpecLike
import java.io.{ByteArrayOutputStream, ObjectOutputStream}
class PersistenceSpec extends AnyWordSpecLike{
case class TestClass(name: String) extends Serializable
def serializeSomething(): ByteArrayOutputStream = {
val testItem = TestClass("My Thing")
val bos: ByteArrayOutputStream = new ByteArrayOutputStream()
val oos = new ObjectOutputStream(bos)
oos.writeObject(testItem)
bos
}
"serializeSomething" when {
"executed" must {
"successfully serialize" in {
val outputStream = serializeSomething()
println(outputStream.toString())
}
}
}
}
When I run this test I get a java.io.NotSerializableException on the call to oos.writeObject(testItem), which makes no sense, since case classes automatically implement Serializable, and this is the simplest possible example.
However, if I paste the code for TestClass and serializeSomething() into repl, I am able to call the function, and it works just fine.
What is different when calling my function via scalatest, vs repl that would cause this exception?
One final note: If I change the call from oos.writeObject(testItem) to oos.writeObject("Hello"), it works fine, even when run from scalatest.
You need to define TestClass outside of PersistenceSpec.
Inner class instances automatically get a reference to the instance of the outer class. So, when you write it out, it tries to serialize the PersistenceSpec instance as well, and that of course fails.
I have a situation where I'm trying to use implicit resolution on a singleton type. This works perfectly fine if I know that singleton type at compile time:
object Main {
type SS = String with Singleton
trait Entry[S <: SS] {
type out
val value: out
}
implicit val e1 = new Entry["S"] {
type out = Int
val value = 3
}
implicit val e2 = new Entry["T"] {
type out = String
val value = "ABC"
}
def resolve[X <: SS](s: X)(implicit x: Entry[X]): x.value.type = {
x.value
}
def main(args: Array[String]): Unit = {
resolve("S") //implicit found! No problem
}
}
However, if I don't know this type at compile time, then I run into issues.
def main(args: Array[String]): Unit = {
val string = StdIn.readLine()
resolve(string) //Can't find implicit because it doesn't know the singleton type at runtime.
}
Is there anyway I can get around this? Maybe some method that takes a String and returns the singleton type of that string?
def getSingletonType[T <: SS](string: String): T = ???
Then maybe I could do
def main(args: Array[String]): Unit = {
val string = StdIn.readLine()
resolve(getSingletonType(string))
}
Or is this just not possible? Maybe you can only do this sort of thing if you know all of the information at compile-time?
If you knew about all possible implementations of Entry in compile time - which would be possible only if it was sealed - then you could use a macro to create a map/partial function String -> Entry[_].
Since this is open to extending, I'm afraid at best some runtime reflection would have to scan the whole classpath to find all possible implementations.
But even then you would have to embed this String literal somehow into each implementations because JVM bytecode knows nothing about mappings between singleton types and implementations - only Scala compiler does. And then use that to find if among all implementations there is one (and exactly one) that matches your value - in case of implicits if there are two of them at once in the same scope compilation would fail, but you can have more than one implementation as long as the don't appear together in the same scope. Runtime reflection would be global so it wouldn't be able to avoid conflicts.
So no, no good solution for making this compile-time dispatch dynamic. You could create such dispatch yourself by e.g. writing a Map[String, Entry[_]] yourself and using get function to handle missing pices.
Normally implicits are resolved at compile time. But val string = StdIn.readLine() becomes known at runtime only. Principally, you can postpone implicit resolution till runtime but you'll be able to apply the results of such resolution at runtime only, not at compile time (static types etc.)
object Entry {
implicit val e1 = ...
implicit val e2 = ...
}
import scala.reflect.runtime.universe._
import scala.reflect.runtime
import scala.tools.reflect.ToolBox
val toolbox = ToolBox(runtime.currentMirror).mkToolBox()
def resolve(s: String): Any = {
val typ = appliedType(
typeOf[Entry[_]].typeConstructor,
internal.constantType(Constant(s))
)
val instanceTree = toolbox.inferImplicitValue(typ, silent = false)
val instance = toolbox.eval(toolbox.untypecheck(instanceTree)).asInstanceOf[Entry[_]]
instance.value
}
resolve("S") // 3
val string = StdIn.readLine()
resolve(string)
// 3 if you enter S
// ABC if you enter T
// "scala.tools.reflect.ToolBoxError: implicit search has failed" otherwise
Please notice that I put implicits into the companion object of type class in order to make them available in the implicit scope and therefore in the toolbox scope. Otherwise the code should be modified slightly:
object EntryImplicits {
implicit val e1 = ...
implicit val e2 = ...
}
// val instanceTree = toolbox.inferImplicitValue(typ, silent = false)
// should be replaced with
val instanceTree =
q"""
import path.to.EntryImplicits._
implicitly[$typ]
"""
In your code import path.to.EntryImplicits._ is import Main._.
Load Dataset from Dynamically generated Case Class
What's wrong with this code:
class Trivials(s:String){
private val x = 0
}
object Trivials {
def main(args: Array[String]): Unit = {
Trivials t = new Trivials("Trivials")
}
}
Both class and object are defined in same source file, hence they are companion.
Error message is as: 'Cannot resolve symbol t'
Wrong syntax (You are using Java syntax) for object creation. In case of Scala you need not mention the type in front of the variable t it will be automatically inferred.
Trivials t = new Trivials("Trivials")
Scala syntax
val t = new Trivials("Trivials")
I'm using Play 2.5 with the Guice dependency injection.
Normal bind works like:
bind(classOf[SomeClass]).to(classOf[DefaultClass])
I now only need to be able to bind a class where the className has been specified in the Configuration.
I tried something like:
val className = config.getString("someClass.className")
val x: Class[_] = Class.forName(className)
bind(classOf[SomeClass]).to(classOf[x])
But then the types are wrong.
It needs to be done via Guice, since the SomeClass has an argument that needs to be injected, otherwise I would have used something like
val className = config.getString("someClass.className")
val x = Class.forName(className).newInstance().asInstanceOf[SomeClass]
bind(classOf[SomeClass]).toInstance(x)
Does anybody have an idea on how to bind it via guice?
I think you are looking for something like this...
lazy val injector = (new GuiceApplicationBuilder).injector()
def inject[T : ClassTag]: T = {
injector.instanceOf[T]
}
This is the simplest version, but doesn't handle arguments. You would create a class and then call this as inject[SomeDep].
I've not found a good way of injecting on-the-fly, the only way of really doing it via #inject at the top most invoked class. We only use inject for unit tests at present.
Thanks for the reply, but I finally got it to work using Scala's Reflection.
object Reflection {
import scala.reflect.api
import reflect.runtime.universe._
import reflect.ClassTag
def classTagToClass[T: reflect.ClassTag]: Class[T] = {
def ctag = implicitly[reflect.ClassTag[T]]
ctag.runtimeClass.asInstanceOf[Class[T]]
}
def typeToClassTag[T: TypeTag]: ClassTag[T] = {
ClassTag[T]( typeTag[T].mirror.runtimeClass( typeTag[T].tpe ) )
}
def stringToTypeTag[A](name: String): TypeTag[A] = {
val c = Class.forName(name) // obtain java.lang.Class object from a string
val mirror = runtimeMirror(c.getClassLoader) // obtain runtime mirror
val sym = mirror.staticClass(name) // obtain class symbol for `c`
val tpe = sym.selfType // obtain type object for `c`
// create a type tag which contains above type object
TypeTag(mirror, new api.TypeCreator {
def apply[U <: api.Universe with Singleton](m: api.Mirror[U]): U#Type =
if (m eq mirror) {
tpe.asInstanceOf[U#Type]
}
else {
throw new IllegalArgumentException(s"Type tag defined in $mirror cannot be migrated to other mirrors.")
}
})
}
}
using the above object you can bind a class using it FQDN in the following way:
configuration.getString("config.className")
.map(className =>
bind(classOf[AbstractClass]).to(classTagToClass(typeToClassTag(stringToTypeTag[AbstractClass](className))))
).getOrElse(bind(classOf[AbstractClass]).to(classOf[AbstractClassImpl]))
I know the benefits of lazy fields when a postponed evaluation of values is needed for some reasons. I was wondering what was the behavior of lazy fields in terms of serialization.
Consider the following class.
class MyClass {
lazy val myLazyVal = {...}
...
}
Questions:
If an instance of MyClass is serialized, does the lazy field get serialized too ?
Does the behavior of serialization change if the field has been accessed or not before the serialization ? I mean, if I don't cause the evaluation of the field, is it considered as null ?
Does the serialization mechanism provoke an implicit evaluation of the lazy field ?
Is there a simple way to avoid the serialization of the variable and getting the value recomputed one more time lazily after the deserialization ? This should happen independently from the evaluation of the field.
Answers
Yes if field was already initialized, if not you can tread it as a method. Value is not computed -> not serialized, but available after de serialization.
If you didn't touch field it's serialized almost as it's a simple 'def' method, you don't need it's type to be serializable itself, it will be recalculated after de-serialization
No
You can add #transient before lazy val definition in my code example, as I understand it will do exactly what you want
Code to prove
object LazySerializationTest extends App {
def serialize(obj: Any): Array[Byte] = {
val bytes = new ByteArrayOutputStream()
val out = new ObjectOutputStream(bytes)
out.writeObject(obj)
out.close()
bytes.toByteArray
}
def deSerialise(bytes: Array[Byte]): MyClass = {
new ObjectInputStream(new ByteArrayInputStream(bytes)).
readObject().asInstanceOf[MyClass]
}
def test(obj: MyClass): Unit = {
val bytes = serialize(obj)
val fromBytes = deSerialise(bytes)
println(s"Original cnt = ${obj.x.cnt}")
println(s"De Serialized cnt = ${fromBytes.x.cnt}")
}
object X {
val cnt = new AtomicInteger()
}
class X {
// Not Serializable
val cnt = X.cnt.incrementAndGet
println(s"Create instance of X #$cnt")
}
class MyClass extends Serializable {
lazy val x = new X
}
// Not initialized
val mc1 = new MyClass
test(mc1)
// Force lazy evaluation
val mc2 = new MyClass
mc2.x
test(mc2) // Failed with NotSerializableException
}