Given a List[Foo], I apply a function f.
def f(foo: Foo): Option[Boolean]
I wrote this function body to see if any Foo evaluates to Some(true).
val optFalse: Option[Boolean] = Some(false)
val foos: List[Foo] = ...
foos.foldLeft(optFalse){
(acc: Option[Boolean], elem: Foo) => {
val isMatch: Option[Boolean] = f(elem)
optionOr(acc)(match) // takes 2 Option[Boolean] and `or`'s them.
}
}
However, the f method could be an expensive operation. I'd prefer to "short-circuit":
foos.foldLeft(optFalse){
(acc: Option[Boolean], elem: Foo) => {
if( isTrueOption(acc) ) acc
else {
val isMatch: Option[Boolean] = f(elem)
isMatch
}
}
}
Is there a non-strict way to fold over the entire List[Foo], but stop evaluation once one of the Option[Boolean] equals Some(true) without my changed code? Perhaps using the => operator?
If so, I'm thinking that there's a "space" cost, but I'm not entirely understanding it. Please include that topic as well in your answer (if there is one).
It's unclear what role Option is playing in your code. If you just want to find out whether there's any value for which f returns Some(true), that sounds like a good use case for exists, which is short-circuit:
foos.exists{x => Some(true) == f(x)}
If you need to know whether any f returns None, then clearly you'll need to evaluate the whole list (at least in the case where they're actually all Some, which is presumably the interesting case).
If it has to be a fold, then fold is strict (it would be possible to write a non-strict fold, but that wouldn't be the standard fold). You could use #Peter's solution in a more structured way by using breakable.
Is this what you are looking for?
foos.reduce {
case (Some(true), _)=> Some(true)
case (_, x) => f(x)
}
Related
Suppose we have Seq val ourSeq = Seq(10,5,3,5,4).
I want to return a new list which reads from the left and stop when it sees a duplicate number (e.g. Seq(10,5,3) since 5 is repeated).
I was thinking of using fold left as such
ourSeq.foldLeft(Seq())(op = (temp, curr) => {
if (!temp.contains(curr)) {
temp :+ curr
} else break
})
but as far as I understand, there is no way to break out of a foldLeft?
Although it can be accomplished with a foldLeft() without any breaking out, I would argue that fold is the wrong tool for the job.
I'm rather fond of unfold(), which was introduced in Scala 2.13.0.
val ourSeq = Seq(10,5,3,5,4)
Seq.unfold((Set.empty[Int],ourSeq)){ case (seen,ns) =>
Option.when(ns.nonEmpty && !seen(ns.head)) {
(ns.head, (seen+ns.head, ns.tail))
}
}
//res0: Seq[Int] = Seq(10, 5, 3)
You are correct that it's not possible to break out of foldLeft. It would theoretically be possible to get the correct result with foldLeft, but you're still going to iterate the whole data structure. It'll be better to use an algorithm that already understands how to terminate early, and since you want to take a prefix, takeWhile will suffice.
import scala.collection.mutable.Set
val ourSeq = Seq(10, 5, 3, 5, 4)
val seen: Set[Int] = Set()
val untilDups = ourSeq.takeWhile((x) => {
if (seen contains x) {
false
} else {
seen += x
true
}
})
print(untilDups)
If you wanted to be totally immutable about this, you could wrap the whole thing in some kind of lazy fold that uses an immutable Set to keep its data. And that's certainly how I'd do it in Haskell. But this is Scala; we have mutability, and we may as well use it locally when it suits us.
This can be done using a recursive function:
def uniquePrefix[T](ourSeq: Seq[T]): List[T] = {
#annotation.tailrec
def loop(rem: List[T], res: List[T]): List[T] =
rem match {
case hd::tail if !res.contains(hd) =>
loop(tail, res :+ hd)
case _ =>
res
}
loop(ourSeq.toList, Nil)
}
This appears more complicated, but once you are familiar with the general pattern recursive functions are simple to write and more powerful than fold operations.
If you are working on large collections, this version is more efficient because it is O(n):
def distinctPrefix[T](ourSeq: Seq[T]): List[T] = {
#annotation.tailrec
def loop(rem: List[T], found: Set[T], res: List[T]): List[T] =
rem match {
case hd::tail if !found.contains(hd) =>
loop(tail, found + hd, hd +: res)
case _ =>
res.reverse
}
loop(ourSeq.toList, Set.empty, Nil)
}
This version works with any Seq and there are other options using Iterator etc. as described in the comments. You would need to be more specific about the type of the collection in order to create an optimised algorithm.
def uniquePrefix[T](ourSeq: Seq[T]): List[T] = {
#annotation.tailrec
def loop(rem: Seq[T], res: List[T]): List[T] =
rem.take(1) match {
case Seq(hd) if !res.contains(hd) =>
loop(rem.drop(1), res :+ hd)
case _ =>
res
}
loop(ourSeq, Nil)
}
Another option you have, is to use the function inits:
ourSeq.inits.dropWhile(curr => curr.distinct.size != curr.size).next()
Code run at Scastie.
How to write an early-return piece of code in scala with no returns/breaks?
For example
for i in 0..10000000
if expensive_operation(i)
return i
return -1
How about
input.find(expensiveOperation).getOrElse(-1)
You can use dropWhile
Here an example:
Seq(2,6,8,3,5).dropWhile(_ % 2 == 0).headOption.getOrElse(default = -1) // -> 8
And here you find more scala-takewhile-example
With your example
(0 to 10000000).dropWhile(!expensive_operation(_)).headOption.getOrElse(default = -1)`
Since you asked for intuition to solve this problem generically. Let me start from the basis.
Scala is (between other things) a functional programming language, as such there is a very important concept for us. And it is that we write programs by composing expressions rather than statements.
Thus, the concept of return value for us means the evaluation of an expression.
(Note this is related to the concept of referential transparency).
val a = expr // a is bounded to the evaluation of expr,
val b = (a, a) // and they are interchangeable, thus b === (expr, expr)
How this relates to your question. In the sense that we really do not have control structures but complex expressions. For example an if
val a = if (expr) exprA else exprB // if itself is an expression, that returns other expressions.
Thus instead of doing something like this:
def foo(a: Int): Int =
if (a != 0) {
val b = a * a
return b
}
return -1
We would do something like:
def foo(a: Int): Int =
if (a != 0)
a * a
else
-1
Because we can bound all the if expression itself as the body of foo.
Now, returning to your specific question. How can we early return a cycle?
The answer is, you can't, at least not without mutations. But, you can use a higher concept, instead of iterating, you can traverse something. And you can do that using recursion.
Thus, let's implement ourselves the find proposed by #Thilo, as a tail-recursive function.
(It is very important that the function is recursive by tail, so the compiler optimizes it as something equivalent to a while loop, that way we will not blow up the stack).
def find(start: Int, end: Int, step: Int = 1)(predicate: Int => Boolean): Option[Int] = {
#annotation.tailrec
def loop(current: Int): Option[Int] =
if (current == end)
None // Base case.
else if (predicate(current))
Some(current) // Early return.
else
loop(current + step) // Recursive step.
loop(current = start)
}
find(0, 10000)(_ == 10)
// res: Option[Int] = Some(10)
Or we may generalize this a little bit more, let's implement find for Lists of any kind of elements.
def find[T](list: List[T])(predicate: T => Boolean): Option[T] = {
#annotation.tailrec
def loop(remaining: List[T]): Option[T] =
remaining match {
case Nil => None
case t :: _ if (predicate(t)) => Some(t)
case _ :: tail => loop(remaining = tail)
}
loop(remaining = list)
}
This is not necessarily the best solution from a practical perspective but I still wanted to add it for educational purposes:
import scala.annotation.tailrec
def expensiveOperation(i: Int): Boolean = ???
#tailrec
def findFirstBy[T](f: (T) => Boolean)(xs: Seq[T]): Option[T] = {
xs match {
case Seq() => None
case Seq(head, _*) if f(head) => Some(head)
case Seq(_, tail#_*) => findFirstBy(f)(tail)
}
}
val result = findFirstBy(expensiveOperation)(Range(0, 10000000)).getOrElse(-1)
Please prefer collections methods (dropWhile, find, ...) in your production code.
There a lot of better answer here but I think a 'while' could work just fine in that situation.
So, this code
for i in 0..10000000
if expensive_operation(i)
return i
return -1
could be rewritten as
var i = 0
var result = false
while(!result && i<(10000000-1)) {
i = i+1
result = expensive_operation(i)
}
After the 'while' the variable 'result' will tell if it succeed or not.
Say I have a function that can take an optional parameter, and I want to return a Some if the argument is None and a None if the argument is Some:
def foo(a: Option[A]): Option[B] = a match {
case Some(_) => None
case None => Some(makeB())
}
So what I want to do is kind of the inverse of map. The variants of orElse are not applicable, because they retain the value of a if it's present.
Is there a more concise way to do this than if (a.isDefined) None else Some(makeB())?
fold is more concise than pattern matching
val op:Option[B] = ...
val inv = op.fold(Option(makeB()))(_ => None)
Overview of this answer:
One-liner solution using fold
Little demo with the fold
Discussion of why the fold-solution could be just as "obvious" as the if-else-solution.
Solution
You can always use fold to transform Option[A] into whatever you want:
a.fold(Option(makeB())){_ => Option.empty[B]}
Demo
Here is a complete runnable example with all the necessary type definitions:
class A
class B
def makeB(): B = new B
def foo(a: Option[A]): Option[B] = a match {
case Some(_) => None
case None => Some(makeB())
}
def foo2(a: Option[A]): Option[B] =
a.fold(Option(makeB())){_ => Option.empty[B]}
println(foo(Some(new A)))
println(foo(None))
println(foo2(Some(new A)))
println(foo2(None))
This outputs:
None
Some(Main$$anon$1$B#5fdef03a)
None
Some(Main$$anon$1$B#48cf768c)
Why fold only seems less intuitive
In the comments, #TheArchetypalPaul has commented that fold seems "lot less obvious" than the if-else solution. I agree, but I still think that it might be interesting to reflect on the reasons why that is.
I think that this is mostly an artifact resulting from the presence of special if-else syntax for booleans.
If there were something like a standard
def ifNone[A, B](opt: Option[A])(e: => B) = new {
def otherwise[C >: B](f: A => C): C = opt.fold((e: C))(f)
}
syntax that can be used like this:
val optStr: Option[String] = Some("hello")
val reversed = ifNone(optStr) {
Some("makeB")
} otherwise {
str => None
}
and, more importantly, if this syntax was mentioned on the first page of every introduction to every programming language invented in the past half-century, then the ifNone-otherwise solution (that is, fold), would look much more natural to most people.
Indeed, the Option.fold method is the eliminator of the Option[T] type: whenever we have an Option[T] and want to get an A out of it, the most obvious thing to expect should be a fold(a)(b) with a: A and b: T => A. In contrast to the special treatment of booleans with the if-else-syntax (which is a mere convention), the fold method is very fundamental, the fact that it must be there can be derived from the first principles.
I've come up with this definition a.map(_ => None).getOrElse(Some(makeB())):
scala> def f[A](a: Option[A]) = a.map(_ => None).getOrElse(Some(makeB()))
f: [A](a: Option[A])Option[makeB]
scala> f(Some(44))
res104: Option[makeB] = None
scala> f(None)
res105: Option[makeB] = Some(makeB())
I think the most concise and clearest might be Option.when(a.isEmpty)(makeB)
If I have Scala tuple Option of the likes:
(Some(1), None)
(None, Some(1))
(None, None)
And I want always to extract always the "Some" value if it exists, and otherwise get the None. The only way with pattern matching?
There is this:
def oneOf[A](tup: (Option[A], Option[A])) = tup._1.orElse(tup._2)
That will return the first option that is defined, or None if neither is.
Edit:
Another way to phrase the same thing is
def oneOf[A](tup: (Option[A], Option[A])) =
tup match { case (first, second) => first.orElse(second) }
It's longer, but perhaps more readable.
This should work:
def f(t: (Option[Int], Option[Int])): Option[Int] = t match {
case (Some(n), _) => Some(n)
case (_, Some(n)) => Some(n)
case _ => None
}
I want always to extract always the Some value if it exists, and otherwise get the None
You can just use orElse
def orOption[T](p: (Option[T], Option[T])): Option[T] = {
val (o1, o2) = p
o1 orElse o2
}
However, this does decide what to do if there exists two Some values:
scala> orOption((Some(1), Some(2)))
res0: Option[Int] = Some(1)
You should probably use pattern matching and then decide what to do if there are two Some values, like throw an exception. Alternatively, consider using a better encoding for the result type than Option.
I want to fold a collection or Y's and return an Option[X]. I want to start with None. Like this...
def f(optX: Option[X], y: Y): Option[X]
val optX = collectionOfY.fold(None) { case (prev, y) => f(prev,y) }
adding unneeded types to make it clearer
val optX: Option[X] = collectionOfY.fold(None) { case (prev: Option[X], y: Y) => f(prev,y) }
However, the compiler can not figure out the type properly and I have to write it like this
val xx: Option[X] = None
val optX = collectionOfY.fold(xx) { case (prev, y) => f(prev,y) }
What is the magic Scala syntax to write this?
Thanks
Peter
Just use foldLeft and any of the following
... foldLeft(Option.empty[X]) ... or ... foldLeft(None: Option[X]) ... or ... foldLeft[Option[X]](None) ...
After all, fold just calls foldLeft. You only really want to use fold when your A1 really is a super-type of A, if that really is the case then you can use fold as above and the compiler will know the type correctly.
For example, Option[List[Int]] <: Option[Seq[Int]] by covariance so we don't get an Any here:
List(Some(List(1,2,3))).fold[Option[Seq[Int]]](None)((_, _) => Some(Seq(1)))
> res2: Option[Seq[Int]] = Some(List(1))
Finally, if you do indeed know Option[X] will be a super-type of Y then say this explicitly in the type declaration of Y - i.e. Y <: Option[X], then you can use fold with the solutions given above.
See When should .empty be used versus the singleton empty instance? for a related discussion.
As was pointed out in the comments above, the preferred solution is change the first parameter passed to Option.fold to avoid the use of None and use Option.empty[X] instead.
val optX = collectionOfY.fold(Option.empty[X]) { case (prev, y) => f(prev,y) }
The Scala compiler accepts this without complaint.
This behavior is logical, because fold is defined as
def fold[A1 >: A](z: A1)(op: (A1, A1) ⇒ A1): A1
that means the starting parameter is a supertype of A, but None is's a supertype of Option[T]. If you specify the type directly (like in your second example) then the compiler has enough information to figure out the return type of fold.
One possible workaround is to specify the result of interaction of head of your collection with None as a starting point, and fold it with the rest of the collection (adding a check for Nil):
val optX = collectionOfY.match {
case Nil => None
case x:xs => xs.fold(f(None,x)) {case (prev,y) => f(prev,y) }
}