I am new to Scala and Akka.
I have the following case class:
case class Demo(userId: String, date: java.util.Date, message: String) extends BusinessModel
I have to send List[Demo] in Json format as response to a get request but I am facing problem in the following code due to Date:
implicit val demoFormat: RootJsonFormat[Demo] = jsonFormat3(Demo)
I would be grateful if you may kindly help me out
You need to provide a format for java.util.Date, since spray doesn't have one by default.
A quick google search leads to https://gist.github.com/owainlewis/ba6e6ed3eb64fd5d83e7 :
import java.text._
import java.util._
import scala.util.Try
import spray.json._
object DateMarshalling {
implicit object DateFormat extends JsonFormat[Date] {
def write(date: Date) = JsString(dateToIsoString(date))
def read(json: JsValue) = json match {
case JsString(rawDate) =>
parseIsoDateString(rawDate)
.fold(deserializationError(s"Expected ISO Date format, got $rawDate"))(identity)
case error => deserializationError(s"Expected JsString, got $error")
}
}
private val localIsoDateFormatter = new ThreadLocal[SimpleDateFormat] {
override def initialValue() = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSX")
}
private def dateToIsoString(date: Date) =
localIsoDateFormatter.get().format(date)
private def parseIsoDateString(date: String): Option[Date] =
Try{ localIsoDateFormatter.get().parse(date) }.toOption
}
Import DateMarshalling._ in piece code where you have wrote implicit val demoFormat: RootJsonFormat[Demo] = jsonFormat3(Demo) and it should be ok now :)
Related
I need to get json from https://api.opendota.com/api/leagues and rebuild it keeping only leagues name and leagues id, save to local file and send this json as a response to client by Get method
package dev.zio.quickstart.dota
import zhttp.http.\*
import zio.\*
import zio.json.\*
import zio.\_
import zio.http.Client
An http app that:
Accepts a `Request` and returns a `Response`
May fail with type of `Throwable`
Uses a `DotaRepo` as the environment
object DotaApp:
def apply(): Http\[DotaRepo, Throwable, Request, Response\] =
Http.collectZIO\[Request\] {
// GET /Leagues
case Method.GET -\> !! / "Leagues" =\>
val url = "https://api.opendota.com/api/leagues"
val program = for {
res \<- Client.request(url)
data \<- res.body.asString
} yield data.map(response =\> Response.json(response.toJson))
}
package dev.zio.quickstart.dota
import java.util.UUID
import zio.json.\*
case class League(name: String, leagueid: Int)
object League:
given JsonEncoder\[League\] =
DeriveJsonEncoder.gen\[League\]
given JsonDecoder\[League\] =
DeriveJsonDecoder.gen\[League\]\`
package dev.zio.quickstart.dota
import zio.\*
trait DotaRepo:
def leagues: Task\[List\[League\]\]
def liveMatches: Task\[List\[LiveMatch\]\]
object DotaRepo:
def leagues: ZIO\[DotaRepo, Throwable, List\[League\]\] =
ZIO.serviceWithZIO\[DotaRepo\](\_.leagues)
def liveMatches: ZIO[DotaRepo, Throwable, List[LiveMatch]] =
ZIO.serviceWithZIO[DotaRepo](_.liveMatches)
package dev.zio.quickstart.dota
import zio.\*
import scala.collection.mutable
case class InmemoryLeague(listBuffer: Ref\[mutable.ListBuffer\[League\]\]) extends DotaRepo:
def getLeagues: UIO\[List\[League\]\] =
listBuffer.get.map(\_.toList)
object InmemoryLeague {
def layer: ZLayer\[Any, Nothing, InmemoryLeague\] =
ZLayer.fromZIO(
Ref.make(mutable.ListBuffer.empty\[League\]).map(new InmemoryLeague(\_))
)
}
I've tried using these examples https://github.com/zio/zio-http/tree/main/zio-http-example/src/main/scala/example
Can't get in to syntax
I'am trying to convert prepareStament(object uses for sending SQL statement to the database ) to Json with scala.
So far, I've discovered that the best way to convert an object to Json in scala is to do it with the net.liftweb library.
But when I tried it, I got an empty json.
this is the code
import java.sql.DriverManager
import net.liftweb.json._
import net.liftweb.json.Serialization.write
object Main {
def main (args: Array[String]): Unit = {
implicit val formats = DefaultFormats
val jdbcSqlConnStr = "sqlserverurl**"
val conn = DriverManager.getConnection(jdbcSqlConnStr)
val statement = conn.prepareStatement("exec select_all")
val piedPierJSON2= write(statement)
println(piedPierJSON2)
}
}
this is the result
{}
I used an object I created , and the conversion worked.
case class Person(name: String, address: Address)
case class Address(city: String, state: String)
val p = Person("Alvin Alexander", Address("Talkeetna", "AK"))
val piedPierJSON3 = write(p)
println(piedPierJSON3)
This is the result
{"name":"Alvin Alexander","address":{"city":"Talkeetna","state":"AK"}}
I understood where the problem was, PrepareStament is an interface, and none of its subtypes are serializable...
I'm going to try to wrap it up and put it in a different class.
I want to be able to filter on a date just like you would in normal SQL. Is that possible? I'm running into an issue on how to convert the string from the text file into a date.
import org.apache.spark._
import org.apache.spark.SparkContext._
import org.apache.spark.sql._
import org.apache.log4j._
import java.text._
//import java.util.Date
import java.sql.Date
object BayAreaBikeAnalysis {
case class Station(ID:Int, name:String, lat:Double, longitude:Double, dockCount:Int, city:String, installationDate:Date)
case class Status(station_id:Int, bikesAvailable:Int, docksAvailable:Int, time:String)
val dateFormat = new SimpleDateFormat("yyyy-MM-dd")
def extractStations(line: String): Station = {
val fields = line.split(",",-1)
val station:Station = Station(fields(0).toInt, fields(1), fields(2).toDouble, fields(3).toDouble, fields(4).toInt, fields(5), dateFormat.parse(fields(6)))
return station
}
def extractStatus(line: String): Status = {
val fields = line.split(",",-1)
val status:Status = Status(fields(0).toInt, fields(1).toInt, fields(2).toInt, fields(3))
return status
}
def main(args: Array[String]) {
// Set the log level to only print errors
//Logger.getLogger("org").setLevel(Level.ERROR)
// Use new SparkSession interface in Spark 2.0
val spark = SparkSession
.builder
.appName("BayAreaBikeAnalysis")
.master("local[*]")
.config("spark.sql.warehouse.dir", "file:///C:/temp")
.getOrCreate()
//Load files into data sets
import spark.implicits._
val stationLines = spark.sparkContext.textFile("Data/station.csv")
val stations = stationLines.map(extractStations).toDS().cache()
val statusLines = spark.sparkContext.textFile("Data/status.csv")
val statuses = statusLines.map(extractStatus).toDS().cache()
//people.select("name").show()
stations.select("installationDate").show()
spark.stop()
}
}
Obviously fields(6).toDate() doesn't compile but I'm not sure what to use.
I think this post is what you are looking for.
Also here you'll find a good tutorial for string parse to date.
Hope this helps!
Following are the ways u can convert string to date in scala.
(1) In case of java.util.date :-
val date= new SimpleDateFormat("yyyy-MM-dd")
date.parse("2017-09-28")
(2) In case of joda's dateTime:-
DateTime.parse("09-28-2017")
Here is a helping function that takes on a string representing a date and transforms it into a Timestamp
import java.sql.Timestamp
import java.util.TimeZone
import java.text.{DateFormat, SimpleDateFormat}
def getTimeStamp(timeStr: String): Timestamp = {
val dateFormat: DateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss")
dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"))
val date: Option[Timestamp] = {
try {
Some(new Timestamp(dateFormat.parse(timeStr).getTime))
} catch {
case _: Exception => Some(Timestamp.valueOf("19700101'T'000000"))
}
}
date.getOrElse(Timestamp.valueOf(timeStr))
}
Obviously, you will need to change your input date format from "yyyy-MM-dd'T'HH:mm:ss" into whatever format you have the date string.
Hope this helps.
I'd like to build a generic method for transforming Scala Case Classes to Mongo Documents.
A promising Document constructor is
fromSeq(ts: Seq[(String, BsonValue)]): Document
I can turn a case class into a Map[String -> Any], but then I've lost the type information I need to use the implicit conversions to BsonValues. Maybe TypeTags can help with this?
Here's what I've tried:
import org.mongodb.scala.bson.BsonTransformer
import org.mongodb.scala.bson.collection.immutable.Document
import org.mongodb.scala.bson.BsonValue
case class Person(age: Int, name: String)
//transform scala values into BsonValues
def transform[T](v: T)(implicit transformer: BsonTransformer[T]): BsonValue = transformer(v)
// turn any case class into a Map[String, Any]
def caseClassToMap(cc: Product) = {
val values = cc.productIterator
cc.getClass.getDeclaredFields.map( _.getName -> values.next).toMap
}
// transform a Person into a Document
def personToDocument(person: Person): Document = {
val map = caseClassToMap(person)
val bsonValues = map.toSeq.map { case (key, value) =>
(key, transform(value))
}
Document.fromSeq(bsonValues)
}
<console>:24: error: No bson implicit transformer found for type Any. Implement or import an implicit BsonTransformer for this type.
(key, transform(value))
def personToDocument(person: Person): Document = {
Document("age" -> person.age, "name" -> person.name)
}
Below code works without manual conversion of an object.
import reactivemongo.api.bson.{BSON, BSONDocument, Macros}
case class Person(name:String = "SomeName", age:Int = 20)
implicit val personHandler = Macros.handler[Person]
val bsonPerson = BSON.writeDocument[Person](Person())
println(s"${BSONDocument.pretty(bsonPerson.getOrElse(BSONDocument.empty))}")
You can use Salat https://github.com/salat/salat. A nice example can be found here - https://gist.github.com/bhameyie/8276017. This is the piece of code that will help you -
import salat._
val dBObject = grater[Artist].asDBObject(artist)
artistsCollection.save(dBObject, WriteConcern.Safe)
I was able to serialize a case class to a BsonDocument using the org.bson.BsonDocumentWriter. The below code runs using scala 2.12 and mongo-scala-driver_2.12 version 2.6.0
My quest for this solution was aided by this answer (where they are trying to serialize in the opposite direction): Serialize to object using scala mongo driver?
import org.mongodb.scala.bson.codecs.Macros
import org.mongodb.scala.bson.codecs.DEFAULT_CODEC_REGISTRY
import org.bson.codecs.configuration.CodecRegistries.{fromRegistries, fromProviders}
import org.bson.codecs.EncoderContext
import org.bson.BsonDocumentWriter
import org.mongodb.scala.bson.BsonDocument
import org.bson.codecs.configuration.CodecRegistry
import org.bson.codecs.Codec
case class Animal(name : String, species: String, genus: String, weight: Int)
object TempApp {
def main(args: Array[String]) {
val jaguar = Animal("Jenny", "Jaguar", "Panthera", 190)
val codecProvider = Macros.createCodecProvider[Animal]()
val codecRegistry: CodecRegistry = fromRegistries(fromProviders(codecProvider), DEFAULT_CODEC_REGISTRY)
val codec = Macros.createCodec[Animal](codecRegistry)
val encoderContext = EncoderContext.builder.isEncodingCollectibleDocument(true).build()
var doc = BsonDocument()
val writr = new BsonDocumentWriter(doc) // need to call new since Java lib w/o companion object
codec.encode(writr, jaguar, encoderContext)
print(doc)
}
};
my application takes in a string like this "2002-10-15 10:55:01.000000". I need to validate inside scala script that the string is a valid for a db2 timestamp.
In general (I'd guess) you would do it moslty the same way as in java with either java.text.DateFormat or joda.time.DateTimeFormat (see Joda time).
A simple example:
import java.text.SimpleDateFormat
import java.util.Date
import scala.util.Try
val date = "2002-10-15 10:55:01.000000"
val formatter = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss.SSSS")
val test = Try[Date](formatter.parse(date))
would give you:
test: scala.util.Try[java.util.Date] = Success(Tue Oct 15 10:55:01 CEST 2002)
Then you could match:
test match {
case Success(date) => // ok
case Failure(exception) => // not ok
}
You should use SimpleDateFormat of java to do that in scala :
object DateParser {
def isValid(f : String, d : String) = {
try {
val format = new java.text.SimpleDateFormat(f)
format.parse(d)
}catch(java.text.ParseException e) {
false
}
}
def main(args : Array[String]) {
val format = "yyyy-MM-dd k:m:s"
println(isValid(format,"2002-10-15 10:55:01.000000"))
println(isValid(format,"2002-10-1510:55:01.000000"))
}
}