In scala, I am giving input jsvalue as:
[
{"priority":1,"whoColumns":{"createdBy":"madhu","createdDate":1606111643373,"lastUpdatedBy":"manojkumar","lastUpdatedDate":1635923035513},"adType":1,"code":"pczpriy","id":3,"uploadDate":1606111560000,"payType":"P"},
{"priority":1,"whoColumns":{"createdBy":"madhu","createdDate":1606111643373,"lastUpdatedBy":"manojkumar","lastUpdatedDate":1635923035513},"adType":1,"code":"pczpriy","id":3,"uploadDate":1606111560000,"payType":"P"}
]
I want to convert this into: Map[String, Any]
but I am getting following error in Postman:
Expected Map as JsObject, but got [{"priority":1,"whoColumns":
And in eclipse I am getting error as Stacktrace:
spray.json.package$.deserializationError(package.scala:23)
Related
I am new to scala and I am trying to write a function that takes in a JSON, converts it to Scala dictionary (Map) and checks for certain keys
Below is part of a function that checks for a bunch keys
import play.api.libs.json.Json
def setParams(jsonString: Map[String, Any]) = {
val paramsMap = Json.parse(jsonString)
if (parmsMap.contains("key_1")) {
println('key_1 present')
}
On compiling it with sbt, I get the following errors
/Users/usr/scala_codes/src/main/scala/wrapped_code.scala:29:26: overloaded method value parse with alternatives:
[error] (input: Array[Byte])play.api.libs.json.JsValue <and>
[error] (input: java.io.InputStream)play.api.libs.json.JsValue <and>
[error] (input: String)play.api.libs.json.JsValue
[error] cannot be applied to (Map[String,Any])
[error] val paramsMap = Json.parse(jsonString)
[error] ^
[error] /Users/usr/scala_codes/src/main/scala/wrapped_code.scala:31:9: not found: value parmsMap
[error] if (parmsMap.contains("key_1")) {
Also, in key-value pairs of the JSON, the keys are all strings but the values could be integers, floats or strings. Do I need to make changes for that?
Seems your input type in setParams function should be String not Map[String, Any]
and you have one typo: if (parmsMap.contains("key_1")) should be if (paramsMap.contains("key_1"))
correct function:
def setParams(jsonString: String): Unit = {
val paramsMap = Json.parse(jsonString).as[Map[String, JsValue]]
if (paramsMap.contains("key_1")) println('key_1 present')
}
case class Response(jobCompleted:String,detailedMessage:String)
override def runJob(sc: HiveContext, runtime: JobEnvironment, data:
JobData): JobOutput = {
val generateResponse= new GenerateResponse(data,sc)
val response=generateResponse.generateResponse()
response.pettyPrint
}
I am trying to get ouput from spark job server in this format from my scala code.
" result":{
"jobCompleted":true,
"detailedMessage":"all good"
}
However what returns to me is the following result:{"{\"jobCompleted\":\"true\",\"detailedMessage.."}.
Can some one please point out what I am doing wrong and how to get the correct format. I also tried response.toJson which returns me the AST format
"result": [{
"jobCompleted": ["true"],
"detailedMessage": ["all good"]
}],
I finally figured it out. Based on this stack over flow question. If there is a better way kindly post here as I am new to scala and spark job server.
Convert DataFrame to RDD[Map] in Scala
So the key is to convert the response to a Map[String,JsValue]. Below is the sample code I was playing with.
case class Response(param1:String,param2:String,param3:List[SubResult])
case class SubResult(lst:List[String])
object ResultFormat extends DefaultJsonProtocol{
implicit val subresultformat=jsonFormat1(SubResult)
implicit val responsefomat=jsonFormat3(Response)
}
type JobOutput=Map[String,JsValue]
def runJob(....){
val xlst=List("one","two")
val ylst=List("three","four")
val subresult1=SubResult(xlst)
val subresult2=SubResult(ylst)
val subResultlist=List(subresult1,subresult2)
val r=Result("xxxx","yyy",subResultlist)
r.toJson.asJsObject.fields
//Returns output type of Map[String,JsValue] which the spark job server serializes correctly.
}
is there an easy way to use datetime/timestamp in scala? What's best practice? I currently use "date" to persist data, but I'd also like to persist the current time.
I'm struggling to set the date. This is my code:
val now = new java.sql.Timestamp(new java.util.Date().getTime)
I also tried to do this:
val now = new java.sql.Date(new java.util.Date().getTime)
When changing the datatype in my evolutions to "timestamp", I got an error:
case class MyObjectModel(
id: Option[Int],
title: String,
createdat: Timestamp,
updatedat: Timestamp,
...)
object MyObjectModel{
implicit val myObjectFormat = Json.format[MyObjectModel]
}
Console:
app\models\MyObjectModel.scala:31: No implicit format for
java.sql.Timestamp available.
[error] implicit val myObjectFormat = Json.format[MyObjectModel]
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
Update:
object ProcessStepTemplatesModel {
implicit lazy val timestampFormat: Format[Timestamp] = new Format[Timestamp] {
override def reads(json: JsValue): JsResult[Timestamp] = json.validate[Long].map(l => Timestamp.from(Instant.ofEpochMilli(l)))
override def writes(o: Timestamp): JsValue = JsNumber(o.getTime)
}
implicit val processStepFormat = Json.format[ProcessStepTemplatesModel]
}
try using this in your code
implicit object timestampFormat extends Format[Timestamp] {
val format = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SS'Z'")
def reads(json: JsValue) = {
val str = json.as[String]
JsSuccess(new Timestamp(format.parse(str).getTime))
}
def writes(ts: Timestamp) = JsString(format.format(ts))
}
it is (de)serialized in a JS compatible format like the following "2018-01-06T18:31:29.436Z"
please note: the implicit object shall be decleared in the code before it is used
I guess your question is handled in What's the standard way to work with dates and times in Scala? Should I use Java types or there are native Scala alternatives?.
Go with Java 8 "java.time".
In the subject you mention Slick (Scala Database Library) but the error you got comes from a Json library and it says that you don't have a converter for java.sql.Timestamp to Json. Without knowing which Json library you are using it's hard to help you with a working example.
I am trying to construct a JSON object from a list where key is "products" and value is List[Product] where Product is a case class.But I am getting error that says "type mismatch; found : (String, List[com.mycompnay.ws.client.Product]) required: net.liftweb.json.JObject (which expands to) net.liftweb.json.JsonAST.JObject".
What I have done so far is as below:
val resultJson:JObject = "products" -> resultList
println(compact(render(resultJson)))
You're looking for decompose (doc). See this answer.
I tested the following code and it worked fine:
import net.liftweb.json._
import net.liftweb.json.JsonDSL._
import net.liftweb.json.Extraction._
implicit val formats = net.liftweb.json.DefaultFormats
case class Product(foo: String)
val resultList: List[Product] = List(Product("bar"), Product("baz"))
val resultJson: JObject = ("products" -> decompose(resultList))
println(compact(render(resultJson)))
Result:
{"products":[{"foo":"bar"},{"foo":"baz"}]}
I'm working on a project to integrate play silhouette with reactivemongo
When I compile my project shows me an error like this:
[error] /reactivemongo-silhouette-rest/app/models/daos/PasswordInfoDAO.scala:83: value getAsTry is not a member of com.mohiva.play.silhouette.api.util.PasswordInfo
[error] Future(Some(fullDoc.getAsTryBSONString.get))
mi method is this:
def find(loginInfo: LoginInfo) = {
implicit val userFormat = Macros.handler[PasswordInfo]
val collection = db[BSONCollection]("PasswordInfo")
val query = BSONDocument(
"loginInfo" -> BSONDocument(
"loginInfo" -> loginInfo.providerID,
"loginInfo" -> loginInfo.providerKey
)
)
val passwordInfo: Future[Option[PasswordInfo]] = collection.find( query ).one[PasswordInfo]
passwordInfo.flatMap {
case None =>
Future.successful(Option.empty[PasswordInfo])
case Some(fullDoc) =>
Future(Some(fullDoc.getAsTry[PasswordInfo]("authInfo").get))
}
//Future.successful(data.get(loginInfo))
}
The complete versión is here:
https://github.com/hectorgool/reactivemongo-silhouette-rest/blob/master/app/models/daos/PasswordInfoDAO.scala
And the PasswordInfo is here:
https://github.com/mohiva/play-silhouette/blob/master/silhouette/app/com/mohiva/play/silhouette/api/util/PasswordHasher.scala
Can someone tell me what am doing wrong or what am i missing?
The repository on github is this:
https://github.com/hectorgool/reactivemongo-silhouette-rest
So by the time you get to your pattern match on passwordInfo its no longer a BSONDocument because of the call to one. You have in scope your userFormat for the type PasswordInfo which provides the implicit reader to the method one and which returns the Future[Option[PasswordInfo]] to your val passwordInfo.
This is why your error appears:
[error] /reactivemongo-silhouette-rest/app/models/daos/PasswordInfoDAO.scala:83: value getAsTry is not a member of com.mohiva.play.silhouette.api.util.PasswordInfo [error]
Because you already have your type [PasswordInfo] and its not a ReactiveMongo type like BSONDocument which has getAsTry defined.
I can't fully determine your return type but hopefully this helps.
Link to ReactiveMongo One method
Link to ReactiveMongo BSONDocument which has getAsTry