combine condition inside flat map and return result - scala

val date2 = Option(LocalDate.parse("2017-02-01"))
//date1.compareTo(date2)>=0
case class dummy(val prop:Seq[Test])
case class Test(val s :String)
case class Result(val s :String)
val s = "11,22,33"
val t = Test(s)
val dt =Test("2017-02-06")
val list = dummy(Seq(t))
val list2 = dummy(Seq(dt))
val code = Option("22")
val f = date2.flatMap(c => list2
.prop
.find(d=>LocalDate.parse(d.s)
.compareTo(c)>=0))
.map(_ => Result("Found"))
.getOrElse(Result("Not Found"))
code.flatMap(c => list
.prop
.find(_.s.split(",").contains(c)))
.map(_ => Result("Found"))
.getOrElse(Result("Not Found"))
I want to && the conditions below and return Result("Found")/Result("Not Found")
d=>LocalDate.parse(d.s).compareTo(c)>=0)
_.s.split(",").contains(c)
Is there any possible way to achieve the above .In actual scenerio list and list 2 are Future

I tried to make a more realistic example based on Futures. Here is how I would do it:
val date2 = Option(LocalDate.parse("2017-02-01"))
case class Test(s: String)
case class Result(s: String)
val t = Test("11,22,33")
val dt = Test("2017-02-06")
val code = Option("22")
val f1 = Future(Seq(t))
val f2 = Future(Seq(dt))
// Wait for both futures to finish
val futureResult = Future.sequence(Seq(f1, f2)).map {
case Seq(s1, s2) =>
// Check the first part, this will be a Boolean
val firstPart = code.nonEmpty && s1.exists(_.s.split(",").contains(code.get))
// Check the second part, also a boolean
val secondPart = date2.nonEmpty && s2.exists(d => LocalDate.parse(d.s).compareTo(date2.get) >= 0)
// Do the AND logic you wanted
if (firstPart && secondPart) {
Result("Found")
} else {
Result("Not Found")
}
}
// This is just for testing to see we got the correct result
val result = Await.result(futureResult, Duration.Inf)
println(result)
As an aside, your code and date2 values in your example are Options... If this is true in your production code, then we should do a check first to see if they are both defined. If they are not then there would be no need to continue with the rest of the code:
val futureResult = if (date2.isEmpty || code.isEmpty) {
Future.successful(Result("Not Found"))
} else {
Future.sequence(Seq(f1, f2)).map {
case Seq(s1, s2) =>
val firstPart = s1.exists(_.s.split(",").contains(code.get))
val secondPart = s2.exists(d => LocalDate.parse(d.s).compareTo(date2.get) >= 0)
if (firstPart && secondPart) {
Result("Found")
} else {
Result("Not Found")
}
}
}

Use pattern matching on Option instead of using flatMap
e.g.
val x = Some("20")
x match {
case Some(i) => println(i) //do whatever you want to do with the value. And then return result
case None => Result("Not Found")
}
Looking at what you are trying to do, You would have to use pattern matching twice, that too nested one.

Related

Efficient way to compare result and send a custom object in scala

val date2 = Option(LocalDate.parse("2017-02-01"))
case class dummy(val prop:Seq[Test])
case class Test(val s :String,val dt:String)
case class Result(val s :String)
def myFunc:Result = {
val s = "11,22,33"
val t = Test(s,"2017-02-06")
val list = dummy(Seq(t))
val code = Option("22")
val result = code.exists(p => {
list.prop.exists(d => d.s.split(",").contains(p) && (LocalDate.parse(d.dt).compareTo(date2.get)>=0))
})
if (result) {
Result("found")
} else {
Result("Not Found")
}
}
The code determines the result based on condition.
Is there a efficient way to achieve the above in scala using map and avoiding date2.get
You should check pattern matching, as far as i can see, you have several cases:
- Code
- list
- date2
One way to avoid date2.get is this one belows:
(code, list, date2) match {
case (Some(p), dummy(l), Some(d2)) if l.exists(d => d.s.split(",").contains(p) && (LocalDate.parse(d.dt).compareTo(d2) >= 0)) => Result("found")
case (_, _, _) => Result("Not Found")
}
Also i don't know why you want to use map. It seems to me that this is not the proper tool for this job

Scala removing IF Else statement and write in a functional way

Let's say I have the following tuple
(colType, colDocV)
Where colType is a boolean and colDocV is a String
Depending on those two values, I will apply some chunk of code that applies transformations to a Dataframe.
Now, this code works. However, I am not convinced this is the proper way to write functional programming code.
I don't know which of these 3 approaches will improve the quality of the code and remove all if-if else-else :
Should I apply some kind of design pattern and which one?
Should I use some kind of pattern matching?
Should I use some anonymous function?
if (colDocV) {
val newCol = udf(UDFHashCode.udfHashCode).apply(col(columnName))
dataframe.withColumn(columnName, newCol)
} else if (colType.contains("string") || colType.contains("text")) {
val newCol = udf(Entropy.stringEntropyFunc).apply(col(columnName)).cast(DoubleType)
dataframe.withColumn(columnName, newCol)
} else if (colType.contains("date")) {
val newCol = udf(DateUtils.getTimeAsDoubleFunc).apply(col(columnName)).cast(DoubleType)
dataframe.withColumn(columnName, newCol)
} else if (colType.contains("long")) {
dataframe.withColumn(columnName, dataframe(columnName).cast(DoubleType) )
} else {
dataframe.drop(columnName) //Dropping column that cannot be processed
}
You can do this with a match statement and a bunch of regexps.
val str = ".*(?:string|text).*".r
val date = ".*date.*".r
val long = ".*long.*".r
def col(tuple: (Boolean, String)) = tuple match {
case (true, _) => Some(udf(...))
case (_, str()) => Some(udf(...))
case (_, date()) => Some(udf(...))
case (, long()) => Some(udf(...))
case _ => None
}
col(colType -> colDocv)
.fold(dataframe.drop(columnName)) { newCol =>
dataframe.withColumn(columnName, newCol)
}
According to what I understand from your question following can be a solution using match case
def callUdf(colDocV: String, colType: Boolean, dataframe: DataFrame) = (colDocV, colType) match {
case x if (x._1.contains("string") || x._1.contains("text")) => dataframe.withColumn(columnName, udf(Entropy.stringEntropyFunc).apply(col(columnName)).cast(DoubleType))
case x if (x._1.contains("date")) => dataframe.withColumn(columnName, udf(DateUtils.getTimeAsDoubleFunc).apply(col(columnName)).cast(DoubleType))
case x if (x._1.contains("long")) => dataframe.withColumn(columnName, dataframe(columnName).cast(DoubleType) )
case _ => dataframe.drop(columnName)
}

Scala - Recursive method is return different values

I have implemented a calculation to obtain the node score of each nodes.
The formula to obtain the value is:
The children list can not be empty or a flag must be true;
The iterative way works pretty well:
class TreeManager {
def scoreNo(nodes:List[Node]): List[(String, Double)] = {
nodes.headOption.map(node => {
val ranking = node.key.toString -> scoreNode(Some(node)) :: scoreNo(nodes.tail)
ranking ::: scoreNo(node.children)
}).getOrElse(Nil)
}
def scoreNode(node:Option[Node], score:Double = 0, depth:Int = 0):Double = {
node.map(n => {
var nodeScore = score
for(child <- n.children){
if(!child.children.isEmpty || child.hasInvitedSomeone == Some(true)){
nodeScore = scoreNode(Some(child), (nodeScore + scala.math.pow(0.5, depth)), depth+1)
}
}
nodeScore
}).getOrElse(score)
}
}
But after i've refactored this piece of code to use recursion, the results are totally wrong:
class TreeManager {
def scoreRecursive(nodes:List[Node]): List[(Int, Double)] = {
def scoreRec(nodes:List[Node], score:Double = 0, depth:Int = 0): Double = nodes match {
case Nil => score
case n =>
if(!n.head.children.isEmpty || n.head.hasInvitedSomeone == Some(true)){
score + scoreRec(n.tail, score + scala.math.pow(0.5, depth), depth + 1)
} else {
score
}
}
nodes.headOption.map(node => {
val ranking = node.key -> scoreRec(node.children) :: scoreRecursive(nodes.tail)
ranking ::: scoreRecursive(node.children)
}).getOrElse(Nil).sortWith(_._2 > _._2)
}
}
The Node is an object of a tree and it's represented by the following class:
case class Node(key:Int,
children:List[Node] = Nil,
hasInvitedSomeone:Option[Boolean] = Some(false))
And here is the part that i'm running to check results:
object Main {
def main(bla:Array[String]) = {
val xx = new TreeManager
val values = List(
Node(10, List(Node(11, List(Node(13))),
Node(12,
List(
Node(14, List(
Node(15, List(Node(18))), Node(17, hasInvitedSomeone = Some(true)),
Node(16, List(Node(19, List(Node(20)))),
hasInvitedSomeone = Some(true))),
hasInvitedSomeone = Some(true))),
hasInvitedSomeone = Some(true))),
hasInvitedSomeone = Some(true)))
val resIterative = xx.scoreNo(values)
//val resRecursive = xx.scoreRec(values)
println("a")
}
}
The iterative way is working because i've checked it but i didn't get why recursive return wrong values.
Any idea?
Thank in advance.
The recursive version never recurses on children of the nodes, just on the tail. Whereas the iterative version correctly both recurse on the children and iterate on the tail.
You'll notice your "iterative" version is also recursive btw.

Can I use Action.async with multiple Futures?

In a previous SO question, I got advice on using Scala Futures with PlayFramework, thank you. Now things have gotten a bit more complicated. Let's say that before I just had to map where fruit could be found:
def getMapData(coll: MongoCollection[Document], s: String): Future[Seq[Document]] = ...
def mapFruit(collection: MongoCollection[Document]) = Action.async {
val fut = getMapData(collection, "fruit")
fut.map { docs: Seq[Document] =>
Ok(docs.toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}
It turns out that people care more about Apples than Bananas or Cherries, so if no more than 100 items should appear on the map, people want Apples to have priority over Bananas and Cherries, but not more than some percentage of items on a map should be Apples. Some function pickDocs determines the proper mix. I thought something like this might just work, but no:
def mapApplesBananasCherries(collection: MongoCollection[Document]) = Action.async {
val futA = getMapData(collection, "apples")
val futB = getMapData(collection, "bananas")
val futC = getMapData(collection, "cherries")
futA.map { docsA: Seq[Document] =>
futB.map { docsB: Seq[Document] =>
futC.map { docsC: Seq[Document] =>
val docsPicked = pickDocs(100, docsA, docsB, docsC)
Ok(docsPicked.toJson)
}
}
// won't compile without something here, e.g. Ok("whatever")
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}
Life was simple when I just had one Future, but now I have three. What can I do to make this to (1) work and (2) again be simple? I can't really construct a web response until all three Futures have values.
Basically, you should use flatMap
futA.flatMap { docsA: Seq[String] =>
futB.flatMap { docsB: Seq[String] =>
futC.map { docsC: Seq[String] =>
docsPicked = pickDocs(100, docsA, docsB, docsC)
Ok(docsPicked.toJson)
}
}
}
Also, you can use for comprehension:
val res = for {
docsA <- futA
docsB <- futB
docsC <- futC
} yield Ok(pickDocs(100, docsA, docsB, docsC).toJson)
res.recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
If my understanding is that you want to execute apples, cherries and bananas in that priority, I would code it similar to this
import scala.concurrent.{Await, Future}
import scala.util.Random
import scala.concurrent.duration._
object WaitingFutures extends App {
implicit val ec = scala.concurrent.ExecutionContext.Implicits.global
val apples = Future {50 + Random.nextInt(100)}
val cherries = Future {50 + Random.nextInt(100)}
val bananas = Future {50 + Random.nextInt(100)}
val mix = for {
app <- apples
cher <- if (app < 100) cherries else Future {0}
ban <- if (app + cher < 100) bananas else Future {0}
} yield (app,cher,ban)
mix.onComplete {m =>
println(s"mix ${m.get}")
}
Await.result(mix, 3 seconds)
}
if apples returns more than 100 when the future completes, it doesn't wait until cherries or bananas are done, but returns a dummy future with 0. If it's not enough it will wait until cherries are executed and so on.
NB I didn't put much effort on how to signal the if, so I'm using the dummy future which might not be the best approach.
This doesn't compile because your nested future block is returning a Future[Future[Future[Response]]]. If you instead use flatMap on the futures, Your futures will not be nested.
If you want this to be a little less repetitive, you can use Future.sequence instead to kick off futures simultaneously. You can either use pattern matching to re-extract the lists:
val futureCollections = List("apples", "bananas", "cherries").map{ getMapData(collection, _) }
Future.sequence(futureCollections) map { case docsA :: docsB :: docsC :: Nil =>
Ok(pickDocs(100, docsA, docsB, docsC).toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
or you could just hand the pickDocs function a list of lists (sorted by priority) for it to pick from.
Future.sequence(futureCollections) map { docLists =>
Ok(pickDocs(docLists, 100, 0.75f).toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
This pickDocs implementation will take a percentage of the head of the list, unless there aren't enough documents in the full list, in which it takes more, then recursively apply the same percentage on the remaining slots lists.
def pickDocs[T](lists: List[List[T]], max: Int, dampPercentage: Float): List[T] = {
lists match {
case Nil => Nil
case head :: tail =>
val remainingLength = tail.flatten.length
val x = max - remainingLength
val y = math.ceil(max * dampPercentage).toInt
val fromHere = head.take(x max y)
fromHere ++ pickDocs(tail, max - fromHere.length, dampPercentage)
}
}
This is a very common pattern for Futures and similar classes that "contain values" (e.g. Option, List)
To combine the results you want to use the flatMap method and the resulting code is
def mapApplesBananasCherries(collection: MongoCollection[Document]) = Action.async {
val futA = getMapData(collection, "apples")
val futB = getMapData(collection, "bananas")
val futC = getMapData(collection, "cherries")
futA.flatMap { docsA =>
futB.flatMap { docsB =>
futC.map { docsC =>
val docsPicked = pickDocs(100, docsA, docsB, docsC)
Ok(docsPicked.toJson)
}
}
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}
In fact it's so common that a special syntax exists to make it more readable, called for-comprehension: the following code is equivalent to the previous snippet
def mapApplesBananasCherries(collection: MongoCollection[Document]) = Action.async {
val futA = getMapData(collection, "apples")
val futB = getMapData(collection, "bananas")
val futC = getMapData(collection, "cherries")
for {
apples <- futA
bananas <- futB
cherries <- futC
} yield {
val docsPicked = pickDocs(100, apples, bananas, cherries)
Ok(docsPicked.toJson)
} recover {
case e => Console.err.println("FAIL: " + e.getMessage); BadRequest("FAIL")
}
}

Tuple seen as Product, compiler rejects reference to element

Constructing phoneVector:
val phoneVector = (
for (i <- 1 until 20) yield {
val p = killNS(r.get("Phone %d - Value" format(i)))
val t = killNS(r.get("Phone %d - Type" format(i)))
if (p == None) None
else
if (t == None) (p,"Main") else (p,t)
}
).filter(_ != None)
Consider this very simple snippet:
for (pTuple <- phoneVector) {
println(pTuple.getClass.getName)
println(pTuple)
//val pKey = pTuple._1.replaceAll("[^\\d]","")
associate() // stub prints "associate"
}
When I run it, I see output like this:
scala.Tuple2
((609) 954-3815,Mobile)
associate
When I uncomment the line with replaceAll(), compile fails:
....scala:57: value _1 is not a member of Product with Serializable
[error] val pKey = pTuple._1.replaceAll("[^\\d]","")
[error] ^
Why does it not recognize pTuple as a Tuple2 and treat it only as Product
OK, this compiles and produces the desired result. But it's too verbose. Can someone please demonstrate a more concise solution for dealing with this typesafe stuff?
for (pTuple <- phoneVector) {
println(pTuple.getClass.getName)
println(pTuple)
val pPhone = pTuple match {
case t:Tuple2[_,_] => t._1
case _ => None
}
val pKey = pPhone match {
case s:String => s.replaceAll("[^\\d]","")
case _ => None
}
println(pKey)
associate()
}
You can do:
for (pTuple <- phoneVector) {
val pPhone = pTuple match {
case (key, value) => key
case _ => None
}
val pKey = pPhone match {
case s:String => s.replaceAll("[^\\d]","")
case _ => None
}
println(pKey)
associate()
}
Or simply phoneVector.map(_._1.replaceAll("[^\\d]",""))
By changing the construction of phoneVector, as wrick's question implied, I've been able to eliminate the match/case stuff because Tuple is assured. Not thrilled by it, but Change is Hard, and Scala seems cool.
Now, it's still possible to slip a None value into either of the Tuple values. My match/case does not check for that, and I suspect that could lead to a runtime error in the replaceAll call. How is that allowed?
def killNS (s:Option[_]) = {
(s match {
case _:Some[_] => s.get
case _ => None
}) match {
case None => None
case "" => None
case s => s
}
}
val phoneVector = (
for (i <- 1 until 20) yield {
val p = killNS(r.get("Phone %d - Value" format(i)))
val t = killNS(r.get("Phone %d - Type" format(i)))
if (t == None) (p,"Main") else (p,t)
}
).filter(_._1 != None)
println(phoneVector)
println(name)
println
// Create the Neo4j nodes:
for (pTuple <- phoneVector) {
val pPhone = pTuple._1 match { case p:String => p }
val pType = pTuple._2
val pKey = pPhone.replaceAll(",.*","").replaceAll("[^\\d]","")
associate(Map("target"->Map("label"->"Phone","key"->pKey,
"dial"->pPhone),
"relation"->Map("label"->"IS_AT","key"->pType),
"source"->Map("label"->"Person","name"->name)
)
)
}
}