I have a model, which has some Option fields, which contain another Option fields. For example:
case class First(second: Option[Second], name: Option[String])
case class Second(third: Option[Third], title: Option[String])
case class Third(numberOfSmth: Option[Int])
I'm receiving this data from external JSON's and sometimes this data may contain null's, that was the reason of such model design.
So the question is: what is the best way to get a deepest field?
First.get.second.get.third.get.numberOfSmth.get
Above method looks really ugly and it may cause exception if one of the objects will be None. I was looking in to Scalaz lib, but didn't figure out a better way to do that.
Any ideas?
The solution is to use Option.map and Option.flatMap:
First.flatMap(_.second.flatMap(_.third.map(_.numberOfSmth)))
Or the equivalent (see the update at the end of this answer):
First flatMap(_.second) flatMap(_.third) map(_.numberOfSmth)
This returns an Option[Int] (provided that numberOfSmth returns an Int). If any of the options in the call chain is None, the result will be None, otherwise it will be Some(count) where count is the value returned by numberOfSmth.
Of course this can get ugly very fast. For this reason scala supports for comprehensions as a syntactic sugar. The above can be rewritten as:
for {
first <- First
second <- first .second
third <- second.third
} third.numberOfSmth
Which is arguably nicer (especially if you are not yet used to seeing map/flatMap everywhere, as will certainly be the case after a while using scala), and generates the exact same code under the hood.
For more background, you may check this other question: What is Scala's yield?
UPDATE:
Thanks to Ben James for pointing out that flatMap is associative. In other words x flatMap(y flatMap z))) is the same as x flatMap y flatMap z. While the latter is usually not shorter, it has the advantage of avoiding any nesting, which is easier to follow.
Here is some illustration in the REPL (the 4 styles are equivalent, with the first two using flatMap nesting, the other two using flat chains of flatMap):
scala> val l = Some(1,Some(2,Some(3,"aze")))
l: Some[(Int, Some[(Int, Some[(Int, String)])])] = Some((1,Some((2,Some((3,aze))))))
scala> l.flatMap(_._2.flatMap(_._2.map(_._2)))
res22: Option[String] = Some(aze)
scala> l flatMap(_._2 flatMap(_._2 map(_._2)))
res23: Option[String] = Some(aze)
scala> l flatMap(_._2) flatMap(_._2) map(_._2)
res24: Option[String] = Some(aze)
scala> l.flatMap(_._2).flatMap(_._2).map(_._2)
res25: Option[String] = Some(aze)
There is no need for scalaz:
for {
first <- yourFirst
second <- f.second
third <- second.third
number <- third.numberOfSmth
} yield number
Alternatively you can use nested flatMaps
This can be done by chaining calls to flatMap:
def getN(first: Option[First]): Option[Int] =
first flatMap (_.second) flatMap (_.third) flatMap (_.numberOfSmth)
You can also do this with a for-comprehension, but it's more verbose as it forces you to name each intermediate value:
def getN(first: Option[First]): Option[Int] =
for {
f <- first
s <- f.second
t <- s.third
n <- t.numberOfSmth
} yield n
I think it is an overkill for your problem but just as a general reference:
This nested access problem is addressed by a concept called Lenses. They provide a nice mechanism to access nested data types by simple composition. As introduction you might want to check for instance this SO answer or this tutorial. The question whether it makes sense to use Lenses in your case is whether you also have to perform a lot of updates in you nested option structure (note: update not in the mutable sense, but returning a new modified but immutable instance). Without Lenses this leads to lengthy nested case class copy code. If you do not have to update at all, I would stick to om-nom-nom's suggestion.
Related
So I've been doing some reading about Monads in scala, and all the syntax relating to their flatMap functions and for comprehensions. Intuitively, I understand why Monads need to use the map part of the flatMap function, as usually when I use a map function its on a container of zero, one or many elements and returns the same container but with the passed in function applied to all the elements of the container. A Monad similarly is a container of zero, one or many elements.
However, what is the purpose of the flatten part of the flatMap? I cannot understand the intuition behind it. To me it seems like extra boilerplate that requires all the functions passed to flatMap to create a container / monad around their return value only to have that container instantly destroyed by the flatten part of the flatMap. I cannot think of a single example where flatMap is used which can't be simplified by being replaced simply by map. For example:
var monad = Option(5)
var monad2 = None
def flatAdder(i:Int) = Option(i + 1)
def adder(i:Int) = i + 1
// What I see in all the examples
monad.flatMap(flatAdder)
// Option[Int] = Some(6)
monad.flatMap(flatAdder).flatMap(flatAdder)
// Option[Int] = Some(7)
monad2.flatMap(flatAdder)
// Option[Int] = None
monad2.flatMap(flatAdder).flatMap(flatAdder)
// Option[Int] = None
// Isn't this a lot easier?
monad.map(adder)
// Option[Int] = Some(6)
monad.map(adder).map(adder)
// Option[Int] = Some(7)
monad2.map(adder)
// Option[Int] = None
monad2.map(adder).map(adder)
// Option[Int] = None
To me, using map by itself seems far more intuitive and simple than flatMap and the flatten part does not seem to add any sort of value. However, in Scala a large emphasis is placed on flatMap rather than map, to the point it even gets its own syntax for for comprehensions, so clearly I must be missing something. My question is: in what cases is the flatten part of flatMap actually useful? and what other advantages does flatMap have over map?
What if each element being processed could result in zero or more elements?
List(4, 0, 15).flatMap(n => primeFactors(n))
//List(2, 2, 3, 5)
What if you have a name, that might not be spelled correctly, and you want their office assignment, if they have one?
def getID(name:String): Option[EmployeeID] = ???
def getOffice(id:EmployeeID): Option[Office] = ???
val office :Option[Office] =
getID(nameAttempt).flatMap(id => getOffice(id))
You use flatMap() when you have a monad and you need to feed its contents to a monad producer. You want the result Monad[X] not Monad[Monad[X]].
The idea behind flatMap is to be able to take a value of M[A] and a function f having a type of A => M[B] and produce an M[B], if we didn't have a way to "flatten" the value then we'd always end up having an M[M[B]] which is what we don't want most of the time.
(Yes, there are some cases that you may want an M[M[_]] like when you want to create multiple instances of M[_])
I have a for comprehension like this:
for {
(value1: String, value2: String, value3: String) <- getConfigs(args)
// more stuff using those values
}
getConfigs returns an Either[Throwable, (Seq[String], String, String)] and when I try to compile I get this error:
value withFilter is not a member of Either[Throwable,(Seq[String], String, String)]
How can I use this method (that returns an Either) in the for comprehension?
Like this:
for {
tuple <- getConfigs()
} println(tuple)
Joking aside, I think that is an interesting question but it is misnamed a bit.
The problem (see above) is not that for comprehensions are not possible but that pattern matching inside the for comprehension is not possible within Either.
There is documentation how for comprehensions are translated but they don't cover each case. This one is not covered there, as far as I can see. So I looked it up in my instance of "Programming in Scala" -- Second Edition (because that is the one I have by my side on dead trees).
Section 23.4 - Translation of for-expressions
There is a subchapter "Translating patterns in generators", which is what is the problem here, as described above. It lists two cases:
Case One: Tuples
Is exactly our case:
for ((x1, …, xn) <- expr1) yield expr2
should translate to expr1.map { case (x1, …, xn) => expr2).
Which is exactly what IntelliJ does, when you select the code and do an "Desugar for comprehension" action. Yay!
… but that makes it even weirder in my eyes, because the desugared code actually runs without problems.
So this case is the one which is (imho) matching the case, but is not what is happening. At least not what we observed. Hm?!
Case two: Arbitrary patterns
for (pat <- expr1) yield expr2
translates to
expr1 withFilter {
case pat => true
case _ => false
} map {
case pat => expr2
}
where there is now an withFilter method!
This case totally explains the error message and why pattern matching in an Either is not possible.
The chapter ultimately refers to the scala language specification (to an older one though) which is where I stop now.
So I a sorry I can't totally answer that question, but hopefully I could hint enough what is the root of the problem here.
Intuition
So why is Either problematic and doesn't propose an withFilter method, where Try and Option do?
Because filter removes elements from the "container" and probably "all", so we need something that is representing an "empty container".
That is easy for Option, where this is obviously None. Also easy for e.g. List. Not so easy for Try, because there are multiple Failure, each one can hold a specific exception. However there are multiple failures taking this place:
NoSuchElementException and
UnsupportedOperationException
and which is why Try[X] runs, but an Either[Throwable, X] does not.
It's almost the same thing, but not entirely. Try knows that Left are Throwable and the library authors can take advantage out of it.
However on an Either (which is now right biased) the "empty" case is the Left case; which is generic. So the user determines which type it is, so the library authors couldn't pick generic instances for each possible left.
I think this is why Either doesn't provide an withFilter out-of-the-box and why your expression fails.
Btw. the
expr1.map { case (x1, …, xn) => expr2) }
case works, because it throws an MatchError on the calling stack and panics out of the problem which… in itself might be a greater problem.
Oh and for the ones that are brave enough: I didn't use the "Monad" word up until now, because Scala doesn't have a datastructure for it, but for-comprehensions work just without it. But maybe a reference won't hurt: Additive Monads have this "zero" value, which is exactly what Either misses here and what I tried to give some meaning in the "intuition" part.
I guess you want your loop to run only if the value is a Right. If it is a Left, it should not run. This can be achieved really easy:
for {
(value1, value2, value3) <- getConfigs(args).right.toOption
// more stuff using those values
}
Sidenote: I don't know whats your exact use case, but scala.util.Try is better suited for cases where you either have a result or a failure (an exception).
Just write Try { /*some code that may throw an exception*/ } and you'll either have Success(/*the result*/) or a Failure(/*the caught exception*/).
If your getConfigs method returns a Try instead of Either, then your above could would work without any changes.
You can do this using Oleg's better-monadic-for compiler plugin:
build.sbt:
addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.2.4")
And then:
object Test {
def getConfigs: Either[Throwable, (String, String, String)] = Right(("a", "b", "c"))
def main(args: Array[String]): Unit = {
val res = for {
(fst, snd, third) <- getConfigs
} yield fst
res.foreach(println)
}
}
Yields:
a
This works because the plugin removes the unnecessary withFilter and unchecked while desugaring and uses a .map call. Thus, we get:
val res: Either[Throwable, String] =
getConfigs
.map[String](((x$1: (String, String, String)) => x$1 match {
case (_1: String, _2: String, _3: String)
(String, String, String)((fst # _), (snd # _), (third # _)) => fst
}));
I think the part you may find surprising is that the Scala compiler emits this error because you deconstruct the tuple in place. This is surprisingly forces the compiler to check for withFilter method because it looks to the compilers like an implicit check for the type of the value inside the container and checks on values are implemented using withFilter. If you write your code as
for {
tmp <- getConfigs(args)
(value1: Seq[String], value2: String, value3: String) = tmp
// more stuff using those values
}
it should compile without errors.
I use sequenceU to turn inside type out while working with disjunctions in scalaz.
for e.g.
val res = List[\/[Errs,MyType]]
doing
res.sequenceU will give \/[Errs,List[MyType]]
Now if I have a val res2 = List[(\/[Errs,MyType], DefModel)] - List containing tuples of disjunctions; what's the right way to convert
res2 to \/[Errs,List[ (Mype,DefModel)]
As noted in the comments, the most straightforward way to write this is probably just with a traverse and map:
def sequence(xs: List[(\/[Errs, MyType], DefModel)]): \/[Errs, List[(MyType, DefModel)]] =
xs.traverseU { case (m, d) => m.map((_, d)) }
It's worth noting, though, that tuples are themselves traversable, so the following is equivalent:
def sequence(xs: List[(\/[Errs, MyType], DefModel)]): \/[Errs, List[(MyType, DefModel)]] =
xs.traverseU(_.swap.sequenceU.map(_.swap))
Note that this would be even simpler if the disjunction were on the right side of the tuple. If you're willing to make that change, you can also more conveniently take advantage of the fact that Traverse instances compose:
def sequence(xs: List[(DefModel, \/[Errs, MyType])]): \/[Errs, List[(DefModel, MyType)]] =
Traverse[List].compose[(DefModel, ?)].sequenceU(xs)
I'm using kind-projector here but you could also write out the type lambda.
take this as a follow up to this SO question
I'm new to scala and working through the 99 problems. The given solution to p9 is:
object P09 {
def pack[A](ls: List[A]): List[List[A]] = {
if (ls.isEmpty) List(List())
else {
val (packed, next) = ls span { _ == ls.head }
if (next == Nil) List(packed)
else packed :: pack(next)
}
}
}
The span function is doing all the work here. As you can see from the API doc (it's the link) span returns a Tuple2 (actually the doc says it returns a pair - but that's been deprecated in favor or Tuple2). I was trying to figure out why you don't get something back like a list-of-lists or some such thing and stumbled across the SO link above. As I understand it, the reason for the Tuple2 has to do with increasing performance by not having to deal with Java's 'boxing/unboxing' of things like ints into objects like Integers. My question is
1) is that an accurate statement?
2) are there other reasons for something like span to return a Tuple2?
thx!
A TupleN object has at least two major differences when compared to a "standard" List+:
(less importantly) the size of the tuple is known beforehand, allowing to better reason about it (by the programmer and the compiler).
(more importantly) a tuple preserves the type information for each of its elements/"slots".
Note that, as alluded, the type Tuple2 is a part of the TupleN family, all utilizing the same concept. For example:
scala> ("1",2,3l)
res0: (String, Int, Long) = (1,2,3)
scala> res0.getClass
res1: Class[_ <: (String, Int, Long)] = class scala.Tuple3
As you can see here, each of the elements in the 3-tuple has a distinct type, allowing for better pattern matching, stricter type protection etc.
+heterogeneous lists are also possible in Scala, but, so far, they're not part of the standard library, and arguably harder to understand, especially for newcomers.
span returns exactly two values. A Tuple2 can hold exactly two values. A list can contain arbitrarily many values. Therefore Tuple2 is just a better fit than using a list.
I am a bit new to Scala, so apologies if this is something a bit trivial.
I have a list of items which I want to iterate through. I to execute a check on each of the items and if just one of them fails I want the whole function to return false. So you can see this as an AND condition. I want it to be evaluated lazily, i.e. the moment I encounter the first false return false.
I am used to the for - yield syntax which filters items generated through some generator (list of items, sequence etc.). In my case however I just want to break out and return false without executing the rest of the loop. In normal Java one would just do a return false; within the loop.
In an inefficient way (i.e. not stopping when I encounter the first false item), I could do it:
(for {
item <- items
if !satisfiesCondition(item)
} yield item).isEmpty
Which is essentially saying that if no items make it through the filter all of them satisfy the condition. But this seems a bit convoluted and inefficient (consider you have 1 million items and the first one already did not satisfy the condition).
What is the best and most elegant way to do this in Scala?
Stopping early at the first false for a condition is done using forall in Scala. (A related question)
Your solution rewritten:
items.forall(satisfiesCondition)
To demonstrate short-circuiting:
List(1,2,3,4,5,6).forall { x => println(x); x < 3 }
1
2
3
res1: Boolean = false
The opposite of forall is exists which stops as soon as a condition is met:
List(1,2,3,4,5,6).exists{ x => println(x); x > 3 }
1
2
3
4
res2: Boolean = true
Scala's for comprehensions are not general iterations. That means they cannot produce every possible result that one can produce out of an iteration, as, for example, the very thing you want to do.
There are three things that a Scala for comprehension can do, when you are returning a value (that is, using yield). In the most basic case, it can do this:
Given an object of type M[A], and a function A => B (that is, which returns an object of type B when given an object of type A), return an object of type M[B];
For example, given a sequence of characters, Seq[Char], get UTF-16 integer for that character:
val codes = for (char <- "A String") yield char.toInt
The expression char.toInt converts a Char into an Int, so the String -- which is implicitly converted into a Seq[Char] in Scala --, becomes a Seq[Int] (actually, an IndexedSeq[Int], through some Scala collection magic).
The second thing it can do is this:
Given objects of type M[A], M[B], M[C], etc, and a function of A, B, C, etc into D, return an object of type M[D];
You can think of this as a generalization of the previous transformation, though not everything that could support the previous transformation can necessarily support this transformation. For example, we could produce coordinates for all coordinates of a battleship game like this:
val coords = for {
column <- 'A' to 'L'
row <- 1 to 10
} yield s"$column$row"
In this case, we have objects of the types Seq[Char] and Seq[Int], and a function (Char, Int) => String, so we get back a Seq[String].
The third, and final, thing a for comprehension can do is this:
Given an object of type M[A], such that the type M[T] has a zero value for any type T, a function A => B, and a condition A => Boolean, return either the zero or an object of type M[B], depending on the condition;
This one is harder to understand, though it may look simple at first. Let's look at something that looks simple first, say, finding all vowels in a sequence of characters:
def vowels(s: String) = for {
letter <- s
if Set('a', 'e', 'i', 'o', 'u') contains letter.toLower
} yield letter.toLower
val aStringVowels = vowels("A String")
It looks simple: we have a condition, we have a function Char => Char, and we get a result, and there doesn't seem to be any need for a "zero" of any kind. In this case, the zero would be the empty sequence, but it hardly seems worth mentioning it.
To explain it better, I'll switch from Seq to Option. An Option[A] has two sub-types: Some[A] and None. The zero, evidently, is the None. It is used when you need to represent the possible absence of a value, or the value itself.
Now, let's say we have a web server where users who are logged in and are administrators get extra javascript on their web pages for administration tasks (like wordpress does). First, we need to get the user, if there's a user logged in, let's say this is done by this method:
def getUser(req: HttpRequest): Option[User]
If the user is not logged in, we get None, otherwise we get Some(user), where user is the data structure with information about the user that made the request. We can then model that operation like this:
def adminJs(req; HttpRequest): Option[String] = for {
user <- getUser(req)
if user.isAdmin
} yield adminScriptForUser(user)
Here it is easier to see the point of the zero. When the condition is false, adminScriptForUser(user) cannot be executed, so the for comprehension needs something to return instead, and that something is the "zero": None.
In technical terms, Scala's for comprehensions provides syntactic sugars for operations on monads, with an extra operation for monads with zero (see list comprehensions in the same article).
What you actually want to accomplish is called a catamorphism, usually represented as a fold method, which can be thought of as a function of M[A] => B. You can write it with fold, foldLeft or foldRight in a sequence, but none of them would actually short-circuit the iteration.
Short-circuiting arises naturally out of non-strict evaluation, which is the default in Haskell, in which most of these papers are written. Scala, as most other languages, is by default strict.
There are three solutions to your problem:
Use the special methods forall or exists, which target your precise use case, though they don't solve the generic problem;
Use a non-strict collection; there's Scala's Stream, but it has problems that prevents its effective use. The Scalaz library can help you there;
Use an early return, which is how Scala library solves this problem in the general case (in specific cases, it uses better optimizations).
As an example of the third option, you could write this:
def hasEven(xs: List[Int]): Boolean = {
for (x <- xs) if (x % 2 == 0) return true
false
}
Note as well that this is called a "for loop", not a "for comprehension", because it doesn't return a value (well, it returns Unit), since it doesn't have the yield keyword.
You can read more about real generic iteration in the article The Essence of The Iterator Pattern, which is a Scala experiment with the concepts described in the paper by the same name.
forall is definitely the best choice for the specific scenario but for illustration here's good old recursion:
#tailrec def hasEven(xs: List[Int]): Boolean = xs match {
case head :: tail if head % 2 == 0 => true
case Nil => false
case _ => hasEven(xs.tail)
}
I tend to use recursion a lot for loops w/short circuit use cases that don't involve collections.
UPDATE:
DO NOT USE THE CODE IN MY ANSWER BELOW!
Shortly after I posted the answer below (after misinterpreting the original poster's question), I have discovered a way superior generic answer (to the listing of requirements below) here: https://stackoverflow.com/a/60177908/501113
It appears you have several requirements:
Iterate through a (possibly large) list of items doing some (possibly expensive) work
The work done to an item could return an error
At the first item that returns an error, short circuit the iteration, throw away the work already done, and return the item's error
A for comprehension isn't designed for this (as is detailed in the other answers).
And I was unable to find another Scala collections pre-built iterator that provided the requirements above.
While the code below is based on a contrived example (transforming a String of digits into a BigInt), it is the general pattern I prefer to use; i.e. process a collection and transform it into something else.
def getDigits(shouldOnlyBeDigits: String): Either[IllegalArgumentException, BigInt] = {
#scala.annotation.tailrec
def recursive(
charactersRemaining: String = shouldOnlyBeDigits
, accumulator: List[Int] = Nil
): Either[IllegalArgumentException, List[Int]] =
if (charactersRemaining.isEmpty)
Right(accumulator) //All work completed without error
else {
val item = charactersRemaining.head
val isSuccess =
item.isDigit //Work the item
if (isSuccess)
//This item's work completed without error, so keep iterating
recursive(charactersRemaining.tail, (item - 48) :: accumulator)
else {
//This item hit an error, so short circuit
Left(new IllegalArgumentException(s"item [$item] is not a digit"))
}
}
recursive().map(digits => BigInt(digits.reverse.mkString))
}
When it is called as getDigits("1234") in a REPL (or Scala Worksheet), it returns:
val res0: Either[IllegalArgumentException,BigInt] = Right(1234)
And when called as getDigits("12A34") in a REPL (or Scala Worksheet), it returns:
val res1: Either[IllegalArgumentException,BigInt] = Left(java.lang.IllegalArgumentException: item [A] is not digit)
You can play with this in Scastie here:
https://scastie.scala-lang.org/7ddVynRITIOqUflQybfXUA