Serialization in Scala - scala

Im trying to understand basics of serialization in Scala. When i run the first example below I get the following output on the last line: res1: A.Mao = A$$anonfun$main$1$Mao$1#78e67e0a
#SerialVersionUID(1L)
class Poo(val aa:Int) extends Serializable {
override def toString() = "Hola"
}
#SerialVersionUID(1L)
class Mao(val hi: Poo) extends Serializable
def serialize() = {
val test = new Mao(new Poo(1))
try{
val fout = new FileOutputStream("c:\\misc\\address.ser");
val oos = new ObjectOutputStream(fout);
oos.writeObject(test);
oos.close();
System.out.println("Done");
}catch {
case ex => ex.printStackTrace();
}
}
serialize()
def ReadObjectFromFile[A](filename: String)(implicit m:scala.reflect.Manifest[A]): A = {
val input = new ObjectInputStream(new FileInputStream(filename))
val obj = input.readObject()
obj match {
case x if m.erasure.isInstance(x) => x.asInstanceOf[A]
case _ => sys.error("Type not what was expected when reading from file")
}
}
ReadObjectFromFile[Mao]("c:\\misc\\address.ser")
If I change the example and use case classes instead things works as expected with the output
res1: A.Mao = Mao(Hola)
case class Poo(val aa:Int) {
override def toString() = "Hola"
}
case class Mao(val hi: Poo)
def serialize() = {
val test = new Mao(new Poo(1))
try{
val fout = new FileOutputStream("c:\\misc\\address.ser");
val oos = new ObjectOutputStream(fout);
oos.writeObject(test);
oos.close();
System.out.println("Done");
}catch {
case ex => ex.printStackTrace();
}
}
def ReadObjectFromFile[A](filename: String)(implicit m:scala.reflect.Manifest[A]): A = {
val input = new ObjectInputStream(new FileInputStream(filename))
val obj = input.readObject()
obj match {
case x if m.erasure.isInstance(x) => x.asInstanceOf[A]
case _ => sys.error("Type not what was expected when reading from file")
}
}
ReadObjectFromFile[Mao]("c:\\misc\\address.ser")
So my questions are:
What do I need to do to get class to give the same output as case class?
Why does case class work without adding any explicit information about serialization?

This has nothing to do with the de/serialization (which seems correct) - it's just the way the result is displayed:
Scala REPL (and Worksheets) use the value's toString method to display it. Case classes override the default toString() method, therefore the output is displayed nicely (as expected). For non-case classes, the defeault implementation of Object.toString() is called, and results in the class name and address that you see.
You can implement toString for the non-case class too, to get the same result:
class Mao(val hi: Poo) extends Serializable {
override def toString = s"Mao($hi)"
}
// program prints:
// Done
// Mao(Hola)

Related

Facing issue while using reflection class concept in scala

I have one main class like this:
class Test {
def exe(first:String, second:String, task:String):String = {
task match {
case "A" => {
val obj = new A(first)
obj.defineSecond(second)
}
case "B" => {
val obj = new B(first)
obj.defineSecond(second)
}
case "C" => {
val obj = new C(first)
obj.defineSecond(second)
}
....so many cases
}
}
}
Instead of writing case in my Test class everytime a new class is added, I tried using the concept of reflection in scala.
Below is what I trying:
val m = ru.runtimeMirror(getClass.getClassLoader)
val classTest = ru.typeOf[Test].typeSymbol.asClass
val cm = m.reflectClass(classTest)
But getting error as "class Test is inner class, use reflectClass on an InstaneMirror to obtain its classMirror".
Can anyone knows how can I can avoid adding cases to my main class everytime a new class is created, instead I can write my main class in a way it will work for every case.
I guess you haven't provided all necessary information in your question. It's written that "class Test is inner class" in your error message but Test is not inner in your code snippet. If you want your runtime-reflection code to be fixed please provide code snippet that reflects actual use case.
Meanwhile you can try a macro (working at compile time)
import scala.language.experimental.macros
import scala.reflect.macros.blackbox
class Test {
def exe(first: String, second: String, task: String): String = macro Test.exeImpl
}
object Test {
def exeImpl(c: blackbox.Context)(first: c.Tree, second: c.Tree, task: c.Tree): c.Tree = {
import c.universe._
val cases = Seq("A", "B", "C").map(name =>
cq"""${Literal(Constant(name))} => {
val obj = new ${TypeName(name)}($first)
obj.defineSecond($second)
}"""
)
q"$task match { case ..$cases }"
}
}
Usage:
class A(s: String) {
def defineSecond(s1: String): String = ""
}
class B(s: String) {
def defineSecond(s1: String): String = ""
}
class C(s: String) {
def defineSecond(s1: String): String = ""
}
new Test().exe("first", "second", "task")
//scalac: "task" match {
// case "A" => {
// val obj = new A("first");
// obj.defineSecond("second")
// }
// case "B" => {
// val obj = new B("first");
// obj.defineSecond("second")
// }
// case "C" => {
// val obj = new C("first");
// obj.defineSecond("second")
// }
//}

How use guave cache loader in F polymorphic code

I have a service, that returns joke from official example:
final case class JokeError(e: Throwable) extends RuntimeException
def impl[F[_] : Sync](C: Client[F]): Jokes[F] = new Jokes[F] {
val dsl = new Http4sClientDsl[F] {}
import dsl._
def get: F[Jokes.Joke] = {
C.expect[Joke](GET(uri"https://icanhazdadjoke.com/"))
.adaptError { case t => JokeError(t) }
}
}
But I want cache first requested joke (just by constant key, this doesn't matter) with guava cache:
object Jokes {
def apply[F[_]](implicit ev: Jokes[F]): Jokes[F] = ev
final case class Joke(joke: String) extends AnyRef
object Joke {
implicit val jokeDecoder: Decoder[Joke] = deriveDecoder[Joke]
implicit def jokeEntityDecoder[F[_]: Sync]: EntityDecoder[F, Joke] =
jsonOf
implicit val jokeEncoder: Encoder[Joke] = deriveEncoder[Joke]
implicit def jokeEntityEncoder[F[_]: Applicative]: EntityEncoder[F, Joke] =
jsonEncoderOf
}
final case class JokeError(e: Throwable) extends RuntimeException
def impl[F[_]: Sync](C: Client[F]): Jokes[F] = new Jokes[F]{
val cacheLoader : CacheLoader[String, Joke] = new CacheLoader[String, Joke] {
override def load(key: String): Joke = {
import dsl._
val joke: F[Joke] = C.expect[Joke](GET(uri"https://icanhazdadjoke.com/"))
.adaptError{ case t => JokeError(t)}
//? F[Joke] => Joke
null
}
}
val cache = CacheBuilder.newBuilder().build(cacheLoader)
val dsl = new Http4sClientDsl[F]{}
def get: F[Jokes.Joke] = {
//it's ok?
cache.get("constant").pure[F]
}
}
}
As you can see, cacheLoader requires "materialized" value F[Joke] => Joke. And cache return pure value without F
How can I use this cache in F polymorpic code?
You're basically asking how to run code polymorphic in F, to do so you need an Effect constraint to your F.
Also instead of using pure, you would need to use delay, since getting a value from the cache is a side effect.
val cacheLoader : CacheLoader[String, Joke] = new CacheLoader[String, Joke] {
override def load(key: String): Joke = {
import dsl._
val joke: F[Joke] = C.expect[Joke](GET(uri"https://icanhazdadjoke.com/"))
.adaptError{ case t => JokeError(t)}
// This is a side effect, but can't avoid it due to the way the API is designed
joke.toIO.unsafeRunSync()
}
}
val cache = CacheBuilder.newBuilder().build(cacheLoader)
val dsl = new Http4sClientDsl[F]{}
def get: F[Jokes.Joke] = {
// This is okay :)
Sync[F].delay(cache.get("constant"))
}
As an aside, if you want to use something that interoperates really well with http4s, I strongly recommend mules. Check it out here:
https://github.com/ChristopherDavenport/mules

File I/O with Free Monads

I have a CSV file that I need to parse and do some action on every record. How do I use Free Monads with it? Currently, I'm loading the entire file into memory and would like to know if there is any better solution. Below is my program:
for {
reader <- F.getReader("my_file.csv")
csvRecords <- C.readCSV(reader)
_ <- I.processCSV(csvRecords)
_ <- F.close(reader)
} yield()
This code works for smaller files, but if I have very large files (over 1 GB), this wouldn't work very well. I'm using Commons CSV for reading the CSVRecords.
Looking into the code at your gist I think that the line with the comment is exactly the line you don't want at all:
object CSVIOInterpreter extends (CSVIO ~> Future) {
import scala.collection.JavaConverters._
override def apply[A](fa: CSVIO[A]): Future[A] = fa match {
case ReadCSV(reader) => Future.fromTry(Try {
CSVFormat.RFC4180
.withFirstRecordAsHeader()
.parse(reader)
.getRecords // Loads the complete file
.iterator().asScala.toStream
})
}
}
Just remove the whole getRecords line. CSVFormat.parse returns an instance of CSVParser which already implements Iterable<CSVRecord>. And the getRecords call is the only thing that force it to read the whole file.
Actually you can see CSVParser.getRecords implementation and it is
public List<CSVRecord> getRecords() throws IOException {
CSVRecord rec;
final List<CSVRecord> records = new ArrayList<>();
while ((rec = this.nextRecord()) != null) {
records.add(rec);
}
return records;
}
So it just materializes the whole file using this.nextRecord call which is obviously a more "core" part of the API.
So when I do a simplified version of your code without the getRecords call:
import cats._
import cats.free.Free
import java.io._
import org.apache.commons.csv._
import scala.collection.JavaConverters._
trait Action[A] {
def run(): A
}
object F {
import Free.liftF
case class GetReader(fileName: String) extends Action[Reader] {
override def run(): Reader = new FileReader(fileName)
}
case class CloseReader(reader: Reader) extends Action[Unit] {
override def run(): Unit = reader.close()
}
def getReader(fileName: String): Free[Action, Reader] = liftF(GetReader(fileName))
def close(reader: Reader): Free[Action, Unit] = liftF(CloseReader(reader))
}
object C {
import Free.liftF
case class ReadCSV(reader: Reader) extends Action[CSVParser] {
override def run(): CSVParser = CSVFormat.DEFAULT.parse(reader)
}
def readCSV(reader: Reader): Free[Action, CSVParser] = liftF(ReadCSV(reader))
}
object I {
import Free.liftF
case class ProcessCSV(parser: CSVParser) extends Action[Unit] {
override def run(): Unit = {
for (r <- parser.asScala)
println(r)
}
}
def processCSV(parser: CSVParser): Free[Action, Unit] = liftF(ProcessCSV(parser))
}
object Runner {
import cats.arrow.FunctionK
import cats.{Id, ~>}
val runner = new (Action ~> Id) {
def apply[A](fa: Action[A]): Id[A] = fa.run()
}
def run[A](free: Free[Action, A]): A = {
free.foldMap(runner)
}
}
def test() = {
val free = for {
// reader <- F.getReader("my_file.csv")
reader <- F.getReader("AssetsImportCompleteSample.csv")
csvRecords <- C.readCSV(reader)
_ <- I.processCSV(csvRecords)
_ <- F.close(reader)
} yield ()
Runner.run(free)
}
it seems to work OK in line-by-line mode.
Here how I use the CSV file to read and do some operation on that -
I use scala.io.Source.fromFile()
I create one case class of the type of header of CSV file to make the data more accessible and operational.
PS: I don't have knowledge of monads, as well as I am in beginner in Scala. I posted this as it may be helpful.
case class AirportData(id:Int, ident:String, name:String, typeAirport:String, latitude_deg:Double,
longitude_deg:Double, elevation_ft:Double, continent:String, iso_country:String, iso_region:String,
municipality:String)
object AirportData extends App {
def toDoubleOrNeg(s: String): Double = {
try {
s.toDouble
} catch {
case _: NumberFormatException => -1
}
}
val source = scala.io.Source.fromFile("resources/airportData/airports.csv")
val lines = source.getLines().drop(1)
val data = lines.flatMap { line =>
val p = line.split(",")
Seq(AirportData(p(0).toInt, p(1).toString, p(2).toString, p(3).toString, toDoubleOrNeg(p(4)), toDoubleOrNeg(p(5)),
toDoubleOrNeg(p(6)), p(7).toString, p(8).toString, p(9).toString, p(10).toString))
}.toArray
source.close()
println(data.length)
data.take(10) foreach println
}

how to write this function the scala way?

Consider this situation, I have a bunch of services that all need to check the input and handle errors.
val log = Logger("access")
def service1(){input=>
try{
val id = input.split(",")(0).toInt
val value = input.split(",")(1)
//do something
} catch {
case e1: NumberFormatException => log.info("number format is wrong")
case e2: ArrayIndexOutOfBoundsException=> log.info("not enough arguments")
}
}
I want to write a method that handles this common part for every service. I could do it this way:
def common(input:String, log:Logger, action:(Int)=>String):String={
try{
val id = input.split(",")(0).toInt
val value = input.split(",")(1)
action(id)
} catch {
case e1: NumberFormatException => log.info("number format is wrong")
case e2: ArrayIndexOutOfBoundsException=> log.info("not enough arguments")
}
}
Then the service function looks like this:
def service1(){input=> common(input, log, id=>{
//do something return a string
})
}
Is there a way to skip the parameters in common so that it looks more elegant like map in collections?
common(id=>{ //... })
import com.typesafe.scalalogging.StrictLogging
class MyService extends AbstractService {
def service1(input: String): String = common(input) {
id =>
id.toString
}
def service2(input: String): String = common(input) {
id =>
id.toString.toLowerCase
}
}
trait AbstractService extends StrictLogging {
def common(input: String)(f: Int => String): String = {
try {
val id = input.split(",")(0).toInt
f(id)
} catch {
case e1: NumberFormatException =>
logger.error("number format is wrong",e1)
"" //???
case e2: ArrayIndexOutOfBoundsException =>
logger.error("not enough arguments",e2)
"" //???
}
}
}
If input is specific you have to put it as input. Otherwise define method def input:String in trait and provide implementation in service.

Scala Macros: Accessing members with quasiquotes

I'm trying to implement an implicit materializer as described here: http://docs.scala-lang.org/overviews/macros/implicits.html
I decided to create a macro that converts a case class from and to a String using quasiquotes for prototyping purposes. For example:
case class User(id: String, name: String)
val foo = User("testid", "foo")
Converting foo to text should result in "testid foo" and vice versa.
Here is the simple trait and its companion object I have created:
trait TextConvertible[T] {
def convertTo(obj: T): String
def convertFrom(text: String): T
}
object TextConvertible {
import language.experimental.macros
import QuasiTest.materializeTextConvertible_impl
implicit def materializeTextConvertible[T]: TextConvertible[T] = macro materializeTextConvertible_impl[T]
}
and here is the macro:
object QuasiTest {
import reflect.macros._
def materializeTextConvertible_impl[T: c.WeakTypeTag](c: Context): c.Expr[TextConvertible[T]] = {
import c.universe._
val tpe = weakTypeOf[T]
val fields = tpe.declarations.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asMethod.accessed
}
val strConvertTo = fields.map {
field => q"obj.$field"
}.reduce[Tree] {
case (acc, elem) => q"""$acc + " " + $elem"""
}
val strConvertFrom = fields.zipWithIndex map {
case (field, index) => q"splitted($index)"
}
val quasi = q"""
new TextConvertible[$tpe] {
def convertTo(obj: $tpe) = $strConvertTo
def convertFrom(text: String) = {
val splitted = text.split(" ")
new $tpe(..$strConvertFrom)
}
}
"""
c.Expr[TextConvertible[T]](quasi)
}
}
which generates
{
final class $anon extends TextConvertible[User] {
def <init>() = {
super.<init>();
()
};
def convertTo(obj: User) = obj.id.$plus(" ").$plus(obj.name);
def convertFrom(text: String) = {
val splitted = text.split(" ");
new User(splitted(0), splitted(1))
}
};
new $anon()
}
The generated code looks fine, but yet I get the error value id in class User cannot be accessed in User in compilation while trying to use the macro.
I suspect I am using a wrong type for fields. I tried field.asMethod.accessed.name, but it results in def convertTo(obj: User) = obj.id .$plus(" ").$plus(obj.name ); (note the extra spaces after id and name), which naturally results in the error value id is not a member of User.
What am I doing wrong?
Ah, figured it out almost immediately after sending my question.
I changed the lines
val fields = tpe.declarations.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asMethod.accessed
}
to
val fields = tpe.declarations.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.name
}
which solved the problem.
The field you get with accessed.name has a special suffix attached to it, to avoid naming conflicts.
The special suffix is scala.reflect.api.StandardNames$TermNamesApi.LOCAL_SUFFIX_STRING, which has the value, you guessed it, a space char.
This is quite evil, of course.