Validation JSON schema runtime - scala

I want to avoid Runtime undefined behaivors as follows:
val jsonExample = Json.toJson(0)
jsonExample.asOpt[Instant]
yield Some(1970-01-01T00:00:00Z)
How can I verify this using partial function with a lift or some other way, to check thatits indeed Instant, or how you recommend to validate?
ex1:
val jsonExample = Json.toJson(Instant.now())
jsonExample match { ... }
ex2:
val jsonExample = Json.toJson(0)
jsonExample match { ... }
Examples for desired output:
validateInstant(Json.toJson(Instant.now())) -> return Some(...)
validateInstant(Json.toJson(0)) -> return None
I can do somthing as follows, maybe some other ideas?
Just wanted to add a note regarding parsing json, there some runtime undefined problems when we are trying to parse .asOpt[T]
for example:
Json.toJson("0").asOpt[BigDecimal] // yields Some(0)
Json.toJson(0).asOpt[Instant] // yields Some(1970-01-01T00:00:00Z)
We can validate it as follows or some other way:
Json.toJson("0") match {
case JsString(value) => Some(value)
case _ => None
}
Json.toJson(0) match {
case JsNumber(value) => Some(value)
case _ => None
}
Json.toJson(Instant.now()) match {
case o # JsString(_) => o.asOpt[Instant]
case _ => None
}

You can use Option:
def filterNumbers[T](value: T)(implicit tjs: Writes[T]): Option[Instant] = {
Option(Json.toJson(value)).filter(_.asOpt[JsNumber].isEmpty).flatMap(_.asOpt[Instant])
}
Then the following:
println(filterNumbers(Instant.now()))
println(filterNumbers(0))
will output:
Some(2021-02-22T10:35:13.777Z)
None

Related

foldLeft vs foldRight vs tailrec on Scala List

Assume I have a collection of c: List[T] sorted.
And I need to make aggregation with foldLeft/foldRight which outputs (List[T],List[T],List[T]), sorted.
To complete this, I have 2 possibilities:
foldLeft, and then reverse each one of the lists (use ::, otherwise its O(n) each step)
foldRight and remain in the same order (use ::)
I know that foldLeft is tailrec implemented and optimized, but making O(n) more steps to reverse the collections, in this use case - which method will generate me better performance?
Assume that the op is O(1)
Example of the code:
def analyzePayload(payload: JsObject, actionRules: List[ActionRule]): InspectorReport = {
val analyzeMode = appConfig.analyzeMode
val (succeed, notApplicable, failed) = actionRules.foldLeft((List[RuleResult](), List[RuleResult](), List[RuleResult]())) { case ( seed # (succeed, notApplicable, failed), actionRule) =>
// Evaluate Single ActionRule
def step(): (List[RuleResult], List[RuleResult], List[RuleResult]) = actionRuleService.eval(actionRule, payload) match {
// If the result is succeed
case EvalResult(true, _, _) => (RuleResult.fromSucceed(actionRule, appConfig.showSucceedAppData) :: succeed, notApplicable, failed)
// If the result is notApplicable
case EvalResult(_, missing #_ :: _, _) => (succeed, RuleResult.fromNotApplicable(actionRule, appConfig.showNotApplicableAppData, missing) :: notApplicable, failed)
// If the result is unmatched
case EvalResult(_, _, unmatched #_ :: _) => (succeed, notApplicable, RuleResult.fromFailed(actionRule, appConfig.showFailedAppData, unmatched) :: failed)
}
analyzeMode match {
case UntilFirstSucceed => if (succeed.isEmpty) step() else seed
case UntilFirstNotApplicable => if (notApplicable.isEmpty) step() else seed
case UntilFirstFailed => if (failed.isEmpty) step() else seed
case Default => step()
case _ => throw new RuntimeException(s"Unknown mode on analyzePayload with mode = ${analyzeMode}")
}
}
InspectorReport(succeed.reverse, notApplicable.reverse, failed.reverse)
}
Before that, I used tailrec which made bad preformance: (why??)
def analyzePayload(payload: JsObject, actionRules: List[ActionRule]): InspectorReport = {
val analyzeMode = appConfig.analyzeMode
def isCompleted(succeed: Int, notApplicable: Int, failed: Int) = {
analyzeMode match {
case Default => false
case UntilFirstSucceed => if (succeed == 0) false else true
case UntilFirstNotApplicable => if (notApplicable == 0) false else true
case UntilFirstFailed => if (failed == 0) false else true
}
}
#tailrec
def analyzePayloadRec(rules: List[ActionRule])(succeed: List[RuleResult], notApplicable: List[RuleResult], failed: List[RuleResult]): (List[RuleResult], List[RuleResult], List[RuleResult]) = {
if (isCompleted(succeed.size, notApplicable.size, failed.size)) (succeed, notApplicable, failed)
else rules match {
// Base cases:
case Nil => (succeed, notApplicable, failed)
// Evaluate case:
case nextRule :: tail =>
actionRuleService.eval(nextRule, payload) match {
// If the result is succeed
case EvalResult(true, _, _) => analyzePayloadRec(tail)(RuleResult.fromSucceed(nextRule, appConfig.showSucceedAppData) :: succeed, notApplicable, failed)
// If the result is notApplicable
case EvalResult(_, missing #_ :: _, _) => analyzePayloadRec(tail)(succeed, RuleResult.fromNotApplicable(nextRule, appConfig.showNotApplicableAppData, missing) :: notApplicable, failed)
// If the result is unmatched
case EvalResult(_, _, unmatched #_ :: _) => analyzePayloadRec(tail)(succeed, notApplicable, RuleResult.fromFailed(nextRule, appConfig.showFailedAppData, unmatched) :: failed)
}
}
}
analyzePayloadRec(actionRules.reverse)(Nil, Nil, Nil).toInspectorReport // todo: if the analyzeModes are not Default - consider use Streams for lazy collection
}
Performance:
Until 22:16, this tailrec analyze was in use, then, the above code.
which you can see that generates much better performance - the same data was in use.
The x-axis is timeline and y-axis is time in ms (1.4s)
Any ideas why? tailrec should be faster, that's why I consider using foldRight.
What makes your tailrec version slow is
isCompleted(succeed.size, notApplicable.size, failed.size)
taking the size of a linked list in Scala is linear in the size, and you're computing the size of all three lists (at least two of which you're never using (and you don't use any of them by Default)).
On the nth iteration of the tailrec, you're going to walk n list nodes between the size computations, so n iterations of tailrec is at least O(n^2).
Passing the lists themselves to isCompleted (as you effectively do in the foldLeft version) and checking isEmpty should dramatically speed things up (especially in the Default case).
Assuming we have those case classes:
case class InspectorReport(succeed: List[ActionRule], notApplicable: List[ActionRule], failed: List[ActionRule])
case class ActionRule()
case class EvalResult(success: Boolean, missing: List[String], unmatched: List[String])
And the evaluator:
class ActionRuleService {
def eval(actionRule: ActionRule, payload: JsObject): EvalResult = EvalResult(true, List(), List())
}
val actionRuleService = new ActionRuleService
I would try something like this:
def analyzePayload(payload: JsObject, actionRules: List[ActionRule]): InspectorReport = {
val evaluated = actionRules.map(actionRule => (actionRule, actionRuleService.eval(actionRule, payload)))
val (success, failed) = evaluated.partition(_._2.success)
val (missing, unmatched) = failed.partition(_._2.missing.nonEmpty)
InspectorReport(success.map(_._1), missing.map(_._1), unmatched.map(_._1))
}
Or:
def analyzePayload(payload: JsObject, actionRules: List[ActionRule]): InspectorReport = {
val evaluated = actionRules.map(actionRule => (actionRule, actionRuleService.eval(actionRule, payload)))
val success = evaluated.collect {
case (actionRule, EvalResult(true, _, _)) =>
actionRule
}
val missing = evaluated.collect {
case (actionRule, EvalResult(false, missing, _)) if missing.nonEmpty =>
actionRule
}
val unmatched = evaluated.collect {
case (actionRule, EvalResult(false, missing, unmatched)) if missing.isEmpty && unmatched.nonEmpty =>
actionRule
}
InspectorReport(success, missing, unmatched)
}

Scala: How to add match vals to a list val

I have a few vals that match for matching values
Here is an example:
val job_ = Try(jobId.toInt) match {
case Success(value) => jobs.findById(value).map(_.id)
.getOrElse( Left(WrongValue("jobId", s"$value is not a valid job id")))
case Failure(_) => jobs.findByName(jobId.toString).map(_.id)
.getOrElse( Left(WrongValue("jobId", s"'$jobId' is not a known job title.")))
}
// Here the value arrives as a string e.i "yes || no || true || or false" then converted to a boolean
val bool_ = bool.toLowerCase() match {
case "yes" => true
case "no" => false
case "true" => true
case "false" => false
case other => Left(Invalid("bool", s"wrong value received"))
}
Note: invalid case is case class Invalid(x: String, xx: String)
above i'm looking for a given job value and checking whether it exist in the db or not,
No I have a few of these and want to add to a list, here is my list val and flatten it:
val errors = List(..all my vals errors...).flatten // <--- my_list_val (how do I include val bool_ and val job_)
if (errors.isEmpty) { do stuff }
My result should contain errors from val bool_ and val job_
THANK!
You need to fix the types first. The type of bool_ is Any. Which does not give you something you can work with.
If you want to use Either, you need to use it everwhere.
Then, the easiest approach would be to use a for comprehension (I am assuming you're dealing with Either[F, T] here, where WrongValue and Invalid are both sub-classes of F and you're not really interested in the errors).
for {
foundJob <- job_
_ <- bool_
} yield {
// do stuff
}
Note, that in Scala >= 2.13 you can use toIntOption when converting the String to Int:
vaj job_: Either[F, T] = jobId.toIntOption match {
case Some(value) => ...
case _ => ...
}
Also, in case expressions, you can use alternatives when you have the same statement for several cases:
val bool_: Either[F, Boolean] = bool.toLowerCase() match {
case "yes" | "true" => Right(true)
case "no" | "false" => Right(false)
case other => Left(Invalid("bool", "wrong value received"))
}
So, according to your question, and your comments, these are the types you're dealing with.
type ID = Long //whatever id is
def WrongValue(x: String, xx: String) :String = "?-?-?"
case class Invalid(x: String, xx: String)
Now let's create a couple of error values.
val job_ :Either[String,ID] = Left(WrongValue("x","xx"))
val bool_ :Either[Invalid,Boolean] = Left(Invalid("x","xx"))
To combine and report them you might do something like this.
val errors :List[String] =
List(job_, bool_).flatMap(_.swap.toOption.map(_.toString))
println(errors.mkString(" & "))
//?-?-? & Invalid(x,xx)
After checking types as #cbley explained. You can just do a filter operation with pattern matching on your list:
val error = List(// your variables ).filter(_ match{
case Left(_) => true
case _ => false
})

Scala return variable type after future is complete Add Comment Collapse

I've got a problem with returning a list after handling futures in scala. My code looks like this:
def getElements(arrayOfIds: Future[Seq[Int]]): Future[Seq[Element]] = {
var elementArray: Seq[Element] = Seq()
arrayOfIds.map {
ids => ids.map(id => dto.getElementById(id).map {
case Some(element) => elementArray = elementArray :+ element
case None => println("Element not found")
})
}
arrayOfIds.onComplete(_ => elementArray)
}
I'd like to do something like .onComplete, however the return type is
Unit and I'd like to return a Future[Seq[Whatever]]. Is there clean way to handle futures like this? Thanks!
Please provide the type of function dto.getElementById. If it is Int => Future[Option[Element]], then:
def getElements(arrayOfIds: Future[Seq[Int]]): Future[Seq[Element]] = {
val allElements: Future[Seq[Option[Element]]] = arrayOfIds.flatMap( ids =>
Future.sequence(ids.map(dto.getElementById))
)
allElements.map(_.flatMap{
case None => println();None
case some => some
})
}
Without logging, it would be:
arrayOfIds.flatMap( ids => Future.traverse(ids.map(dto.getElementById))(_.flatten))
Instead of assigning the result to a mutable variable, return it from the continuation of the Future. You can use flatMap to extract only the Element results which actually contain a value:
def getElements(arrayOfIds: Future[Seq[Int]]): Future[Seq[Element]] = {
arrayOfIds.flatMap(id => Future.fold(id.map(getElementById))(Seq.empty[Element])(_ ++ _))
}

Tuple seen as Product, compiler rejects reference to element

Constructing phoneVector:
val phoneVector = (
for (i <- 1 until 20) yield {
val p = killNS(r.get("Phone %d - Value" format(i)))
val t = killNS(r.get("Phone %d - Type" format(i)))
if (p == None) None
else
if (t == None) (p,"Main") else (p,t)
}
).filter(_ != None)
Consider this very simple snippet:
for (pTuple <- phoneVector) {
println(pTuple.getClass.getName)
println(pTuple)
//val pKey = pTuple._1.replaceAll("[^\\d]","")
associate() // stub prints "associate"
}
When I run it, I see output like this:
scala.Tuple2
((609) 954-3815,Mobile)
associate
When I uncomment the line with replaceAll(), compile fails:
....scala:57: value _1 is not a member of Product with Serializable
[error] val pKey = pTuple._1.replaceAll("[^\\d]","")
[error] ^
Why does it not recognize pTuple as a Tuple2 and treat it only as Product
OK, this compiles and produces the desired result. But it's too verbose. Can someone please demonstrate a more concise solution for dealing with this typesafe stuff?
for (pTuple <- phoneVector) {
println(pTuple.getClass.getName)
println(pTuple)
val pPhone = pTuple match {
case t:Tuple2[_,_] => t._1
case _ => None
}
val pKey = pPhone match {
case s:String => s.replaceAll("[^\\d]","")
case _ => None
}
println(pKey)
associate()
}
You can do:
for (pTuple <- phoneVector) {
val pPhone = pTuple match {
case (key, value) => key
case _ => None
}
val pKey = pPhone match {
case s:String => s.replaceAll("[^\\d]","")
case _ => None
}
println(pKey)
associate()
}
Or simply phoneVector.map(_._1.replaceAll("[^\\d]",""))
By changing the construction of phoneVector, as wrick's question implied, I've been able to eliminate the match/case stuff because Tuple is assured. Not thrilled by it, but Change is Hard, and Scala seems cool.
Now, it's still possible to slip a None value into either of the Tuple values. My match/case does not check for that, and I suspect that could lead to a runtime error in the replaceAll call. How is that allowed?
def killNS (s:Option[_]) = {
(s match {
case _:Some[_] => s.get
case _ => None
}) match {
case None => None
case "" => None
case s => s
}
}
val phoneVector = (
for (i <- 1 until 20) yield {
val p = killNS(r.get("Phone %d - Value" format(i)))
val t = killNS(r.get("Phone %d - Type" format(i)))
if (t == None) (p,"Main") else (p,t)
}
).filter(_._1 != None)
println(phoneVector)
println(name)
println
// Create the Neo4j nodes:
for (pTuple <- phoneVector) {
val pPhone = pTuple._1 match { case p:String => p }
val pType = pTuple._2
val pKey = pPhone.replaceAll(",.*","").replaceAll("[^\\d]","")
associate(Map("target"->Map("label"->"Phone","key"->pKey,
"dial"->pPhone),
"relation"->Map("label"->"IS_AT","key"->pType),
"source"->Map("label"->"Person","name"->name)
)
)
}
}

How to match a string on a prefix and get the rest?

I can write the code like this:
str match {
case s if s.startsWith("!!!") => s.stripPrefix("!!!")
case _ =>
}
But I want to know is there any better solutions. For example:
str match {
case "!!!" + rest => rest
case _ =>
}
val r = """^!!!(.*)""".r
val r(suffix) = "!!!rest of string"
So suffix will be populated with rest of string, or a scala.MatchError gets thrown.
A different variant would be:
val r = """^(!!!){0,1}(.*)""".r
val r(prefix,suffix) = ...
And prefix will either match the !!! or be null. e.g.
(prefix, suffix) match {
case(null, s) => "No prefix"
case _ => "Prefix"
}
The above is a little more complex than you might need, but it's worth looking at the power of Scala's regexp integration.
Starting Scala 2.13, it's now possible to pattern match a String by unapplying a string interpolator:
"!!!hello" match {
case s"!!!$rest" => rest
case _ => "oups"
}
// "hello"
If it's the sort of thing you do often, it's probably worth creating an extractor
object BangBangBangString{
def unapply(str:String):Option[String]= {
str match {
case s if s.startsWith("!!!") => Some(s.stripPrefix("!!!"))
case _ => None
}
}
}
Then you can use the extractor as follows
str match{
case BangBangBangString(rest) => println(rest)
case _ => println("Doesn't start with !!!")
}
or even
for(BangBangBangString(rest)<-myStringList){
println("rest")
}
Good question !
Even i was trying a lot to find out the answer.
Here is a good link where I found the answer
object _04MatchExpression_PatternGuards {
def main(args: Array[String]): Unit = {
val url: String = "Jan";
val monthType = url match {
case url if url.endsWith(".org") => "Educational Websites";
case url if url.endsWith(".com") => "Commercial Websites";
case url if url.endsWith(".co.in") => "Indian Websites"
case _ => "Unknow Input";
}
}
}