So I've been doing some reading about Monads in scala, and all the syntax relating to their flatMap functions and for comprehensions. Intuitively, I understand why Monads need to use the map part of the flatMap function, as usually when I use a map function its on a container of zero, one or many elements and returns the same container but with the passed in function applied to all the elements of the container. A Monad similarly is a container of zero, one or many elements.
However, what is the purpose of the flatten part of the flatMap? I cannot understand the intuition behind it. To me it seems like extra boilerplate that requires all the functions passed to flatMap to create a container / monad around their return value only to have that container instantly destroyed by the flatten part of the flatMap. I cannot think of a single example where flatMap is used which can't be simplified by being replaced simply by map. For example:
var monad = Option(5)
var monad2 = None
def flatAdder(i:Int) = Option(i + 1)
def adder(i:Int) = i + 1
// What I see in all the examples
monad.flatMap(flatAdder)
// Option[Int] = Some(6)
monad.flatMap(flatAdder).flatMap(flatAdder)
// Option[Int] = Some(7)
monad2.flatMap(flatAdder)
// Option[Int] = None
monad2.flatMap(flatAdder).flatMap(flatAdder)
// Option[Int] = None
// Isn't this a lot easier?
monad.map(adder)
// Option[Int] = Some(6)
monad.map(adder).map(adder)
// Option[Int] = Some(7)
monad2.map(adder)
// Option[Int] = None
monad2.map(adder).map(adder)
// Option[Int] = None
To me, using map by itself seems far more intuitive and simple than flatMap and the flatten part does not seem to add any sort of value. However, in Scala a large emphasis is placed on flatMap rather than map, to the point it even gets its own syntax for for comprehensions, so clearly I must be missing something. My question is: in what cases is the flatten part of flatMap actually useful? and what other advantages does flatMap have over map?
What if each element being processed could result in zero or more elements?
List(4, 0, 15).flatMap(n => primeFactors(n))
//List(2, 2, 3, 5)
What if you have a name, that might not be spelled correctly, and you want their office assignment, if they have one?
def getID(name:String): Option[EmployeeID] = ???
def getOffice(id:EmployeeID): Option[Office] = ???
val office :Option[Office] =
getID(nameAttempt).flatMap(id => getOffice(id))
You use flatMap() when you have a monad and you need to feed its contents to a monad producer. You want the result Monad[X] not Monad[Monad[X]].
The idea behind flatMap is to be able to take a value of M[A] and a function f having a type of A => M[B] and produce an M[B], if we didn't have a way to "flatten" the value then we'd always end up having an M[M[B]] which is what we don't want most of the time.
(Yes, there are some cases that you may want an M[M[_]] like when you want to create multiple instances of M[_])
Related
This question already has answers here:
Is there a difference between foreach and map?
(9 answers)
Closed 5 years ago.
i tried two ways for making a list out of something
var response = List[RS_TxnNested]
consumertxnlist.foreach(txData => {
response = RS_TxnNested(blabla) +: response
})
where consumertxnlist is a Seq[something] .
Another way is
var response = consumerTxnList._2.map(txData => RS_TxnNested(blabla))
Can someone help me in clarifying which one is better and why?
As you've noticed already, the map function in the Seq trait returns a value. Its signature in fact is
def map[B](f: (A) ⇒ B): Seq[B]
It is designed to apply a function to every element of a collection extending the Seq trait and return a new collection.
On the other hand, foreach, in the same trait, has the following signature:
def foreach(f: (A) ⇒ Unit): Unit
This means that it's designed to execute functions with side-effects.
In your example, you're treating the operation of assigning the value to "response" as a side effect. If you're actual goal is to return a value (like it seems you want to do), then use map.
The second version is more explicit in what it does.
map conveys a meaning of transforming something, which is exactly what you're doing.
foreach instead performs an operation on something outside its scope, which is not as clear.
In other words, when I read foreach I have to look at the whole scope, because I know you'll be performing a side effect (since foreach returns Unit)
When I read map I can confidently assume the only expression involved is the one I'm currently looking at.
map lowers the burden on the reader, and makes the code easier to reason about.
Using the foreach and the prepending of the elements will yield a List with elements in reversed order.
When using foreach you have to define response as a var, while using map you can define response as a val.
I would suggest to use map since it better resembles your intent.
Using foreach returns Unit (nothing) which can be used if you're doing side effects or something like that.
If you want to return data then you should use things like map, flatMap, fold and others.
scala> val l: List[String] = List("a", "b", "c")
l: List[String] = List(a, b, c)
scala> l.map(e => e.toUpperCase(java.util.Locale.ROOT))
res0: List[String] = List(A, B, C)
scala> l.foreach(e => println(e))
a
b
c
If you want to return the object from your loop operation , I think you should avoid forEach . Rather go for foldLeft or some other kind of fold operation. As far as your question is concerned , once again if you are returning response then you should go with map else forEach should work fine.
I use sequenceU to turn inside type out while working with disjunctions in scalaz.
for e.g.
val res = List[\/[Errs,MyType]]
doing
res.sequenceU will give \/[Errs,List[MyType]]
Now if I have a val res2 = List[(\/[Errs,MyType], DefModel)] - List containing tuples of disjunctions; what's the right way to convert
res2 to \/[Errs,List[ (Mype,DefModel)]
As noted in the comments, the most straightforward way to write this is probably just with a traverse and map:
def sequence(xs: List[(\/[Errs, MyType], DefModel)]): \/[Errs, List[(MyType, DefModel)]] =
xs.traverseU { case (m, d) => m.map((_, d)) }
It's worth noting, though, that tuples are themselves traversable, so the following is equivalent:
def sequence(xs: List[(\/[Errs, MyType], DefModel)]): \/[Errs, List[(MyType, DefModel)]] =
xs.traverseU(_.swap.sequenceU.map(_.swap))
Note that this would be even simpler if the disjunction were on the right side of the tuple. If you're willing to make that change, you can also more conveniently take advantage of the fact that Traverse instances compose:
def sequence(xs: List[(DefModel, \/[Errs, MyType])]): \/[Errs, List[(DefModel, MyType)]] =
Traverse[List].compose[(DefModel, ?)].sequenceU(xs)
I'm using kind-projector here but you could also write out the type lambda.
I am a bit new to Scala, so apologies if this is something a bit trivial.
I have a list of items which I want to iterate through. I to execute a check on each of the items and if just one of them fails I want the whole function to return false. So you can see this as an AND condition. I want it to be evaluated lazily, i.e. the moment I encounter the first false return false.
I am used to the for - yield syntax which filters items generated through some generator (list of items, sequence etc.). In my case however I just want to break out and return false without executing the rest of the loop. In normal Java one would just do a return false; within the loop.
In an inefficient way (i.e. not stopping when I encounter the first false item), I could do it:
(for {
item <- items
if !satisfiesCondition(item)
} yield item).isEmpty
Which is essentially saying that if no items make it through the filter all of them satisfy the condition. But this seems a bit convoluted and inefficient (consider you have 1 million items and the first one already did not satisfy the condition).
What is the best and most elegant way to do this in Scala?
Stopping early at the first false for a condition is done using forall in Scala. (A related question)
Your solution rewritten:
items.forall(satisfiesCondition)
To demonstrate short-circuiting:
List(1,2,3,4,5,6).forall { x => println(x); x < 3 }
1
2
3
res1: Boolean = false
The opposite of forall is exists which stops as soon as a condition is met:
List(1,2,3,4,5,6).exists{ x => println(x); x > 3 }
1
2
3
4
res2: Boolean = true
Scala's for comprehensions are not general iterations. That means they cannot produce every possible result that one can produce out of an iteration, as, for example, the very thing you want to do.
There are three things that a Scala for comprehension can do, when you are returning a value (that is, using yield). In the most basic case, it can do this:
Given an object of type M[A], and a function A => B (that is, which returns an object of type B when given an object of type A), return an object of type M[B];
For example, given a sequence of characters, Seq[Char], get UTF-16 integer for that character:
val codes = for (char <- "A String") yield char.toInt
The expression char.toInt converts a Char into an Int, so the String -- which is implicitly converted into a Seq[Char] in Scala --, becomes a Seq[Int] (actually, an IndexedSeq[Int], through some Scala collection magic).
The second thing it can do is this:
Given objects of type M[A], M[B], M[C], etc, and a function of A, B, C, etc into D, return an object of type M[D];
You can think of this as a generalization of the previous transformation, though not everything that could support the previous transformation can necessarily support this transformation. For example, we could produce coordinates for all coordinates of a battleship game like this:
val coords = for {
column <- 'A' to 'L'
row <- 1 to 10
} yield s"$column$row"
In this case, we have objects of the types Seq[Char] and Seq[Int], and a function (Char, Int) => String, so we get back a Seq[String].
The third, and final, thing a for comprehension can do is this:
Given an object of type M[A], such that the type M[T] has a zero value for any type T, a function A => B, and a condition A => Boolean, return either the zero or an object of type M[B], depending on the condition;
This one is harder to understand, though it may look simple at first. Let's look at something that looks simple first, say, finding all vowels in a sequence of characters:
def vowels(s: String) = for {
letter <- s
if Set('a', 'e', 'i', 'o', 'u') contains letter.toLower
} yield letter.toLower
val aStringVowels = vowels("A String")
It looks simple: we have a condition, we have a function Char => Char, and we get a result, and there doesn't seem to be any need for a "zero" of any kind. In this case, the zero would be the empty sequence, but it hardly seems worth mentioning it.
To explain it better, I'll switch from Seq to Option. An Option[A] has two sub-types: Some[A] and None. The zero, evidently, is the None. It is used when you need to represent the possible absence of a value, or the value itself.
Now, let's say we have a web server where users who are logged in and are administrators get extra javascript on their web pages for administration tasks (like wordpress does). First, we need to get the user, if there's a user logged in, let's say this is done by this method:
def getUser(req: HttpRequest): Option[User]
If the user is not logged in, we get None, otherwise we get Some(user), where user is the data structure with information about the user that made the request. We can then model that operation like this:
def adminJs(req; HttpRequest): Option[String] = for {
user <- getUser(req)
if user.isAdmin
} yield adminScriptForUser(user)
Here it is easier to see the point of the zero. When the condition is false, adminScriptForUser(user) cannot be executed, so the for comprehension needs something to return instead, and that something is the "zero": None.
In technical terms, Scala's for comprehensions provides syntactic sugars for operations on monads, with an extra operation for monads with zero (see list comprehensions in the same article).
What you actually want to accomplish is called a catamorphism, usually represented as a fold method, which can be thought of as a function of M[A] => B. You can write it with fold, foldLeft or foldRight in a sequence, but none of them would actually short-circuit the iteration.
Short-circuiting arises naturally out of non-strict evaluation, which is the default in Haskell, in which most of these papers are written. Scala, as most other languages, is by default strict.
There are three solutions to your problem:
Use the special methods forall or exists, which target your precise use case, though they don't solve the generic problem;
Use a non-strict collection; there's Scala's Stream, but it has problems that prevents its effective use. The Scalaz library can help you there;
Use an early return, which is how Scala library solves this problem in the general case (in specific cases, it uses better optimizations).
As an example of the third option, you could write this:
def hasEven(xs: List[Int]): Boolean = {
for (x <- xs) if (x % 2 == 0) return true
false
}
Note as well that this is called a "for loop", not a "for comprehension", because it doesn't return a value (well, it returns Unit), since it doesn't have the yield keyword.
You can read more about real generic iteration in the article The Essence of The Iterator Pattern, which is a Scala experiment with the concepts described in the paper by the same name.
forall is definitely the best choice for the specific scenario but for illustration here's good old recursion:
#tailrec def hasEven(xs: List[Int]): Boolean = xs match {
case head :: tail if head % 2 == 0 => true
case Nil => false
case _ => hasEven(xs.tail)
}
I tend to use recursion a lot for loops w/short circuit use cases that don't involve collections.
UPDATE:
DO NOT USE THE CODE IN MY ANSWER BELOW!
Shortly after I posted the answer below (after misinterpreting the original poster's question), I have discovered a way superior generic answer (to the listing of requirements below) here: https://stackoverflow.com/a/60177908/501113
It appears you have several requirements:
Iterate through a (possibly large) list of items doing some (possibly expensive) work
The work done to an item could return an error
At the first item that returns an error, short circuit the iteration, throw away the work already done, and return the item's error
A for comprehension isn't designed for this (as is detailed in the other answers).
And I was unable to find another Scala collections pre-built iterator that provided the requirements above.
While the code below is based on a contrived example (transforming a String of digits into a BigInt), it is the general pattern I prefer to use; i.e. process a collection and transform it into something else.
def getDigits(shouldOnlyBeDigits: String): Either[IllegalArgumentException, BigInt] = {
#scala.annotation.tailrec
def recursive(
charactersRemaining: String = shouldOnlyBeDigits
, accumulator: List[Int] = Nil
): Either[IllegalArgumentException, List[Int]] =
if (charactersRemaining.isEmpty)
Right(accumulator) //All work completed without error
else {
val item = charactersRemaining.head
val isSuccess =
item.isDigit //Work the item
if (isSuccess)
//This item's work completed without error, so keep iterating
recursive(charactersRemaining.tail, (item - 48) :: accumulator)
else {
//This item hit an error, so short circuit
Left(new IllegalArgumentException(s"item [$item] is not a digit"))
}
}
recursive().map(digits => BigInt(digits.reverse.mkString))
}
When it is called as getDigits("1234") in a REPL (or Scala Worksheet), it returns:
val res0: Either[IllegalArgumentException,BigInt] = Right(1234)
And when called as getDigits("12A34") in a REPL (or Scala Worksheet), it returns:
val res1: Either[IllegalArgumentException,BigInt] = Left(java.lang.IllegalArgumentException: item [A] is not digit)
You can play with this in Scastie here:
https://scastie.scala-lang.org/7ddVynRITIOqUflQybfXUA
I have a model, which has some Option fields, which contain another Option fields. For example:
case class First(second: Option[Second], name: Option[String])
case class Second(third: Option[Third], title: Option[String])
case class Third(numberOfSmth: Option[Int])
I'm receiving this data from external JSON's and sometimes this data may contain null's, that was the reason of such model design.
So the question is: what is the best way to get a deepest field?
First.get.second.get.third.get.numberOfSmth.get
Above method looks really ugly and it may cause exception if one of the objects will be None. I was looking in to Scalaz lib, but didn't figure out a better way to do that.
Any ideas?
The solution is to use Option.map and Option.flatMap:
First.flatMap(_.second.flatMap(_.third.map(_.numberOfSmth)))
Or the equivalent (see the update at the end of this answer):
First flatMap(_.second) flatMap(_.third) map(_.numberOfSmth)
This returns an Option[Int] (provided that numberOfSmth returns an Int). If any of the options in the call chain is None, the result will be None, otherwise it will be Some(count) where count is the value returned by numberOfSmth.
Of course this can get ugly very fast. For this reason scala supports for comprehensions as a syntactic sugar. The above can be rewritten as:
for {
first <- First
second <- first .second
third <- second.third
} third.numberOfSmth
Which is arguably nicer (especially if you are not yet used to seeing map/flatMap everywhere, as will certainly be the case after a while using scala), and generates the exact same code under the hood.
For more background, you may check this other question: What is Scala's yield?
UPDATE:
Thanks to Ben James for pointing out that flatMap is associative. In other words x flatMap(y flatMap z))) is the same as x flatMap y flatMap z. While the latter is usually not shorter, it has the advantage of avoiding any nesting, which is easier to follow.
Here is some illustration in the REPL (the 4 styles are equivalent, with the first two using flatMap nesting, the other two using flat chains of flatMap):
scala> val l = Some(1,Some(2,Some(3,"aze")))
l: Some[(Int, Some[(Int, Some[(Int, String)])])] = Some((1,Some((2,Some((3,aze))))))
scala> l.flatMap(_._2.flatMap(_._2.map(_._2)))
res22: Option[String] = Some(aze)
scala> l flatMap(_._2 flatMap(_._2 map(_._2)))
res23: Option[String] = Some(aze)
scala> l flatMap(_._2) flatMap(_._2) map(_._2)
res24: Option[String] = Some(aze)
scala> l.flatMap(_._2).flatMap(_._2).map(_._2)
res25: Option[String] = Some(aze)
There is no need for scalaz:
for {
first <- yourFirst
second <- f.second
third <- second.third
number <- third.numberOfSmth
} yield number
Alternatively you can use nested flatMaps
This can be done by chaining calls to flatMap:
def getN(first: Option[First]): Option[Int] =
first flatMap (_.second) flatMap (_.third) flatMap (_.numberOfSmth)
You can also do this with a for-comprehension, but it's more verbose as it forces you to name each intermediate value:
def getN(first: Option[First]): Option[Int] =
for {
f <- first
s <- f.second
t <- s.third
n <- t.numberOfSmth
} yield n
I think it is an overkill for your problem but just as a general reference:
This nested access problem is addressed by a concept called Lenses. They provide a nice mechanism to access nested data types by simple composition. As introduction you might want to check for instance this SO answer or this tutorial. The question whether it makes sense to use Lenses in your case is whether you also have to perform a lot of updates in you nested option structure (note: update not in the mutable sense, but returning a new modified but immutable instance). Without Lenses this leads to lengthy nested case class copy code. If you do not have to update at all, I would stick to om-nom-nom's suggestion.
In Python, I can do something like this:
lazy = ((i,j) for i in range(0,10000) for j in range(0,10000))
sum((1 for i in lazy))
It will take a while, but the memory use is constant.
The same construct in scala:
(for(i<-0 to 10000; j<-i+1 to 10000) yield (i,j)).count((a:(Int,Int)) => true)
After a while, I get a java.lang.OutOfMemoryError, even though it should be evaluated lazily.
Nothing's inherently lazy about Scala's for-comprehension; it's syntactic sugar* which won't change the fact that the combination of your two ranges will be eager.
If you work with lazy views of your ranges, the result of the comprehension will be lazy too:
scala> for(i<-(0 to 10000).view; j<-(i+1 to 10000).view) yield (i,j)
res0: scala.collection.SeqView[(Int, Int),Seq[_]] = SeqViewN(...)
scala> res0.count((a: (Int, Int)) => true)
res1: Int = 50005000
The laziness here is nothing to do with the for-comprehension, but because when flatMap or map (see below) are called on some type of container, you get back a result in the same type of container. So, the for-comprehension will just preserve the laziness (or lack of) of whatever you put in.
*for something like:
(0 to 10000).flatMap(i => (i+1 to 10000).map(j => (i, j)))
Laziness comes not from the for-comprehension, but from the collection itself. You should look into the strictness characteristics of the collection.
But, for the lazy :-), here's a summary: Iterator and Stream are non-strict, as are selected methods of the view of any collection. So, if you want laziness, be sure to .iterator, .view or .toStream your collection first.