Java Object serialization in scala - scala

Question

Pardon me as I am new to Scala.
I have created a case class which encapsultes some information. One of the objects i want to take in for that is of JavaClass. As i am using in spark, i would need it to be serializable. How can i do that?
                              
Java class
public class Currency {
public Currency(final BigDecimal amount, final CurrencyUnit unit) {
//Doing Something
}
}
case class ReconEntity(inputCurrency : Currency, outputCurrency : Currency)
Using implicit i want to have my serialization code for Currency so that spark can work on ReconEntity.

Answer 1

Firstly, have you tried some RDD operations using your Currency and ReconEntity classes? Do you actually get an error? Spark is able to handle RDD operations with apparently non-serializable Scala classes as values, at least (you can try this in the spark-shell, though possibly this might require the Kryo serializer to be enabled).
Since you state that you don't own the Currency class, you can't add extends Serializable, which would be the simplest solution.
Another approach is to wrap the class with a serializable wrapper, as described in this article: Beating Serialization in Spark - example code copied here for convenience:
For simple classes, it is easiest to make a wrapper interface that
extends Serializable. This means that even though UnserializableObject
cannot be serialized we can pass in the following object without any
issue
public interface UnserializableWrapper extends Serializable {
public UnserializableObject create(String prm1, String prm2);
}
The object can then be passed into an RDD or Map function using the
following approach
UnserializableWrapper usw = new UnserializableWrapper() {
public UnserializableObject create(String prm1, String prm2) {
return new UnserializableObject(prm1,prm2);
} }
If the class is merely a data structure, without significant methods, then it might be easier to unpack its fields into your RDD types (in your case, ReconEntity) and discard the class itself.
If the class has methods that you need, then your other (ugly) option is to cut-and-paste code into a new serializable class or into helper functions in your Spark code.