I'm trying to combine two Option[Iterable[_]] into a new Option[Iterable[_]]. I would like to return a Some if one (or both) of the elements is a Some and a None otherwise. It seems like there should be an idiomatic way of doing this, but I can't seem to find one. The following seems to do what I want, but isn't quite the slick solution I was hoping for.
def merge(
i1: Option[Iterable[_]], i2: Option[Iterable[_]]
): Option[Iterable[_]] = (i1, i2) match {
case (Some(as), Some(bs)) => Some(as ++ bs)
case (a # Some(as), None) => a
case (None, b # Some(bs)) => b
case _ => None
}
Any tips are appreciated. Thanks!
If you're willing to put up with a bit of abstract algebra, there's a nice generalization here: Iterable[_] is a monoid under concatenation, where a monoid's just a set of things (iterable collections, in this case) and an addition-like operation (concatenation) with some simple properties and an identity element (the empty collection).
Similarly, if A is a monoid, then Option[A] is also a monoid under a slightly more general version of your merge:
Some(xs) + Some(ys) == Some(xs + ys)
Some(xs) + None == Some(xs)
None + Some(ys) == Some(ys)
None + None == None
(Note that we need the fact that A is a monoid to know what to do in the first line.)
The Scalaz library captures all these generalizations in its Monoid type class, which lets you write your merge like this:
import scalaz._, Scalaz._
def merge(i1: Option[Iterable[_]], i2: Option[Iterable[_]]) = i1 |+| i2
Which works as expected:
scala> merge(Some(1 to 5), None)
res0: Option[Iterable[_]] = Some(Range(1, 2, 3, 4, 5))
scala> merge(Some(1 to 5), Some(4 :: 3 :: 2 :: 1 :: Nil))
res1: Option[Iterable[_]] = Some(Vector(1, 2, 3, 4, 5, 4, 3, 2, 1))
scala> merge(None, None)
res2: Option[Iterable[_]] = None
(Note that there are other operations that would give valid Monoid instances for Iterable and Option, but yours are the most commonly used, and the ones that Scalaz provides by default.)
This works:
def merge(i1: Option[Iterable[_]], i2: Option[Iterable[_]]): Option[Iterable[_]] =
(for (a <- i1; b <- i2) yield a ++ b).orElse(i1).orElse(i2)
The for/yield portion will add the contents of the options if and only if both are Some.
You can also drop some of the dots and parentheses if you want:
(for (a <- i1; b <- i2) yield a ++ b) orElse i1 orElse i2
You could use this for arbitrary arity:
def merge(xs: Option[Iterable[_]]*) =
if (xs.forall(_.isEmpty)) None else Some(xs.flatten.flatten)
Related
I would like to do almost exactly this in scala. Is there an elegant way?
Specifically, I just want the difference of adjacent elements in a sequence. For example
input = 1,2,6,9
output = 1,4,3
How about this?
scala> List(1, 2, 6, 9).sliding(2).map { case Seq(x, y, _*) => y - x }.toList
res0: List[Int] = List(1, 4, 3)
Here is one that uses recursion and works best on Lists
def differences(l:List[Int]) : List[Int] = l match {
case a :: (rest # b :: _) => (b - a) :: differences(rest)
case _ => Nil
}
And here is one that should be pretty fast on Vector or Array:
def differences(a:IndexedSeq[Int]) : IndexedSeq[Int] =
a.indices.tail.map(i => a(i) - a(i-1))
Of course there is always this:
def differences(a:Seq[Int]) : Seq[Int] =
a.tail.zip(a).map { case (x,y) => x - y }
Note that only the recursive version handles empty lists without an exception.
Why does this construction cause a Type Mismatch error in Scala?
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
<console>:6: error: type mismatch;
found : List[(Int, Int)]
required: Option[?]
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
If I switch the Some with the List it compiles fine:
for (first <- List(1,2,3); second <- Some(1)) yield (first,second)
res41: List[(Int, Int)] = List((1,1), (2,1), (3,1))
This also works fine:
for (first <- Some(1); second <- Some(2)) yield (first,second)
For comprehensions are converted into calls to the map or flatMap method. For example this one:
for(x <- List(1) ; y <- List(1,2,3)) yield (x,y)
becomes that:
List(1).flatMap(x => List(1,2,3).map(y => (x,y)))
Therefore, the first loop value (in this case, List(1)) will receive the flatMap method call. Since flatMap on a List returns another List, the result of the for comprehension will of course be a List. (This was new to me: For comprehensions don't always result in streams, not even necessarily in Seqs.)
Now, take a look at how flatMap is declared in Option:
def flatMap [B] (f: (A) ⇒ Option[B]) : Option[B]
Keep this in mind. Let's see how the erroneous for comprehension (the one with Some(1)) gets converted to a sequence of map calls:
Some(1).flatMap(x => List(1,2,3).map(y => (x, y)))
Now, it's easy to see that the parameter of the flatMap call is something that returns a List, but not an Option, as required.
In order to fix the thing, you can do the following:
for(x <- Some(1).toSeq ; y <- List(1,2,3)) yield (x, y)
That compiles just fine. It is worth noting that Option is not a subtype of Seq, as is often assumed.
An easy tip to remember, for comprehensions will try to return the type of the collection of the first generator, Option[Int] in this case. So, if you start with Some(1) you should expect a result of Option[T].
If you want a result of List type, you should start with a List generator.
Why have this restriction and not assume you'll always want some sort of sequence? You can have a situation where it makes sense to return Option. Maybe you have an Option[Int] that you want to combine with something to get a Option[List[Int]], say with the following function: (i:Int) => if (i > 0) List.range(0, i) else None; you could then write this and get None when things don't "make sense":
val f = (i:Int) => if (i > 0) Some(List.range(0, i)) else None
for (i <- Some(5); j <- f(i)) yield j
// returns: Option[List[Int]] = Some(List(0, 1, 2, 3, 4))
for (i <- None; j <- f(i)) yield j
// returns: Option[List[Int]] = None
for (i <- Some(-3); j <- f(i)) yield j
// returns: Option[List[Int]] = None
How for comprehensions are expanded in the general case are in fact a fairly general mechanism to combine an object of type M[T] with a function (T) => M[U] to get an object of type M[U]. In your example, M can be Option or List. In general it has to be the same type M. So you can't combine Option with List. For examples of other things that can be M, look at subclasses of this trait.
Why did combining List[T] with (T) => Option[T] work though when you started with the List? In this case the library use a more general type where it makes sense. So you can combine List with Traversable and there is an implicit conversion from Option to Traversable.
The bottom line is this: think about what type you want the expression to return and start with that type as the first generator. Wrap it in that type if necessary.
It probably has something to do with Option not being an Iterable. The implicit Option.option2Iterable will handle the case where compiler is expecting second to be an Iterable. I expect that the compiler magic is different depending on the type of the loop variable.
I always found this helpful:
scala> val foo: Option[Seq[Int]] = Some(Seq(1, 2, 3, 4, 5))
foo: Option[Seq[Int]] = Some(List(1, 2, 3, 4, 5))
scala> foo.flatten
<console>:13: error: Cannot prove that Seq[Int] <:< Option[B].
foo.flatten
^
scala> val bar: Seq[Seq[Int]] = Seq(Seq(1, 2, 3, 4, 5))
bar: Seq[Seq[Int]] = List(List(1, 2, 3, 4, 5))
scala> bar.flatten
res1: Seq[Int] = List(1, 2, 3, 4, 5)
scala> foo.toSeq.flatten
res2: Seq[Int] = List(1, 2, 3, 4, 5)
Since Scala 2.13 Option was made IterableOnce
sealed abstract class Option[+A] extends IterableOnce[A] with Product with Serializable
so the following for comprehension works without the use of option2Iterable implicit conversion
scala> for {
| a <- List(1)
| b <- Some(41)
| } yield (a + b)
val res35: List[Int] = List(42)
scala> List(1).flatMap
final override def flatMap[B](f: Int => scala.collection.IterableOnce[B]): List[B]
where we see List#flatMap takes a function to IterableOnce. Desugaring above for comprehension we get something like
List(1).flatMap(a => Some(41).map(b => a + b))
which show the absence of the implicit conversion.
However in Scala 2.12 and before Option was not a traversable/iterable entity
sealed abstract class Option[+A] extends Product with Serializable
so the above for comprehension would desugar to something like
List(1).flatMap(a => option2Iterable(Some(41)).map(b => a + b))(List.canBuildFrom[Int])
where we see the implicit conversion.
The reason it does not work the other way around where for comprehension begins with Option and then we try to chain a List
scala> for {
| a <- Option(1)
| b <- List(41)
| } yield (a + b)
b <- List(41)
^
On line 3: error: type mismatch;
found : List[Int]
required: Option[?]
scala> Option(1).flatMap
final def flatMap[B](f: Int => Option[B]): Option[B]
is because Option#flatMap takes a function to Option and converting a List to Option probably does not make sense because we would lose elements for Lists with more than one element.
As szeiger explains
I think the recent Option changes actually make the for comprehensions
use case easier to understand because you do not need an implicit
conversion anymore. Option can be used on the RHS of a flatMap of any
collection type because it is IterableOnce (but not the opposite
because the RHS of Option#flatMap requires Option).
What is the best way to compute a function on each element in a collection with an incremental shadowing of each element, like this simple example :
val v = IndexedSeq(1,2,3,4)
v.shadowMap{ e => e + 1}
shadow 1: (3,4,5)
shadow 2: (2,4,5)
shadow 3: (2,3,5)
shadow 4: (2,3,4)
I think first to patch or slice to make this, but perhaps there is a more better way to do that in pure functional style ?
Thanks
Sr.
def shadowMap[A,B](xs: Seq[A])(f: A => B) = {
val ys = xs map f
for (i <- ys.indices; (as, bs) = ys splitAt i) yield as ++ bs.tail
}
You can define it like this:
class ShadowMapSeq[A, Repr <: Seq[A]](seq: SeqLike[A, Repr]) {
def shadowMap[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): Iterator[That] = {
seq.indices.iterator.map { i =>
val b = bf(seq.asInstanceOf[Repr])
b.sizeHint(seq.size - 1)
b ++= (seq.take(i) ++ seq.drop(i + 1)).map(f)
b.result
}
}
}
implicit def shadowMapSeq[A, Repr <: Seq[A]](seq: SeqLike[A, Repr]) = new ShadowMapSeq(seq)
And then use it like this:
scala> val v = IndexedSeq(1, 2, 3, 4)
scala> val results = v.shadowMap(_ + 1)
scala> results foreach println
Vector(3, 4, 5)
Vector(2, 4, 5)
Vector(2, 3, 5)
Vector(2, 3, 4)
If by "pure functional style" you mean something like "without making reference to indices" (since you say you want to avoid patch and slice), you can do this pretty elegantly with zippers. For example, here's how to write it with Scalaz's zipper implementation (this is just a demonstration—if you want to wrap it up more nicely you can use the approach dhg gives in his answer):
import scalaz._, Scalaz._
List(1, 2, 3, 4).map(_ + 1).toZipper.map(
_.positions.map(p => (p.lefts.reverse ++ p.rights).toList).toStream
).flatten
In general you're probably better off going with dhg's solution in real code, but the zipper is a handy data structure to know about, and it's a good fit for this problem.
Scalaz provides a method named fold for various ADTs such as Boolean, Option[_], Validation[_, _], Either[_, _] etc. This method basically takes functions corresponding to all possible cases for that given ADT. In other words, a pattern match shown below:
x match {
case Case1(a, b, c) => f(a, b, c)
case Case2(a, b) => g(a, b)
.
.
case CaseN => z
}
is equivalent to:
x.fold(f, g, ..., z)
Some examples:
scala> (9 == 8).fold("foo", "bar")
res0: java.lang.String = bar
scala> 5.some.fold(2 *, 2)
res1: Int = 10
scala> 5.left[String].fold(2 +, "[" +)
res2: Any = 7
scala> 5.fail[String].fold(2 +, "[" +)
res6: Any = 7
At the same time, there is an operation with the same name for the Traversable[_] types, which traverses over the collection performing certain operation on its elements, and accumulating the result value. For example,
scala> List(2, 90, 11).foldLeft("Contents: ")(_ + _.toString + " ")
res9: java.lang.String = "Contents: 2 90 11 "
scala> List(2, 90, 11).fold(0)(_ + _)
res10: Int = 103
scala> List(2, 90, 11).fold(1)(_ * _)
res11: Int = 1980
Why are these two operations identified with the same name - fold/catamorphism? I fail to see any similarities/relation between the two. What am I missing?
I think the problem you are having is that you see these things based on their implementation, not their types. Consider this simple representation of types:
List[A] = Nil
| Cons head: A tail: List[A]
Option[A] = None
| Some el: A
Now, let's consider Option's fold:
fold[B] = (noneCase: => B, someCase: A => B) => B
So, on Option, it reduces every possible case to some value in B, and return that. Now, let's see the same thing for List:
fold[B] = (nilCase: => B, consCase: (A, List[A]) => B) => B
Note, however, that we have a recursive call there, on List[A]. We have to fold that somehow, but we know fold[B] on a List[A] will always return B, so we can rewrite it like this:
fold[B] = (nilCase: => B, consCase: (A, B) => B) => B
In other words, we replaced List[A] by B, because folding it will always return a B, given the type signature of fold. Now, let's see Scala's (use case) type signature for foldRight:
foldRight[B](z: B)(f: (A, B) ⇒ B): B
Say, does that remind you of something?
If you think of "folding" as "condensing all the values in a container through an operation, with a seed value", and you think of an Option as a container that can can have at most one value, then this starts to make sense.
In fact, foldLeft has the same signature and gives you exactly the same results if you use it on an empty list vs None, and on a list with only one element vs Some:
scala> val opt : Option[Int] = Some(10)
opt: Option[Int] = Some(10)
scala> val lst : List[Int] = List(10)
lst: List[Int] = List(10)
scala> opt.foldLeft(1)((a, b) => a + b)
res11: Int = 11
scala> lst.foldLeft(1)((a, b) => a + b)
res12: Int = 11
fold is also defined on both List and Option in the Scala standard library, with the same signature (I believe they both inherit it from a trait, in fact). And again, you get the same results on a singleton list as on Some:
scala> opt.fold(1)((a, b) => a * b)
res25: Int = 10
scala> lst.fold(1)((a, b) => a * b)
res26: Int = 10
I'm not 100% sure about the fold from Scalaz on Option/Either/etc, you raise a good point there. It seems to have quite a different signature and operation from the "folding" I'm used to.
Why does this construction cause a Type Mismatch error in Scala?
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
<console>:6: error: type mismatch;
found : List[(Int, Int)]
required: Option[?]
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
If I switch the Some with the List it compiles fine:
for (first <- List(1,2,3); second <- Some(1)) yield (first,second)
res41: List[(Int, Int)] = List((1,1), (2,1), (3,1))
This also works fine:
for (first <- Some(1); second <- Some(2)) yield (first,second)
For comprehensions are converted into calls to the map or flatMap method. For example this one:
for(x <- List(1) ; y <- List(1,2,3)) yield (x,y)
becomes that:
List(1).flatMap(x => List(1,2,3).map(y => (x,y)))
Therefore, the first loop value (in this case, List(1)) will receive the flatMap method call. Since flatMap on a List returns another List, the result of the for comprehension will of course be a List. (This was new to me: For comprehensions don't always result in streams, not even necessarily in Seqs.)
Now, take a look at how flatMap is declared in Option:
def flatMap [B] (f: (A) ⇒ Option[B]) : Option[B]
Keep this in mind. Let's see how the erroneous for comprehension (the one with Some(1)) gets converted to a sequence of map calls:
Some(1).flatMap(x => List(1,2,3).map(y => (x, y)))
Now, it's easy to see that the parameter of the flatMap call is something that returns a List, but not an Option, as required.
In order to fix the thing, you can do the following:
for(x <- Some(1).toSeq ; y <- List(1,2,3)) yield (x, y)
That compiles just fine. It is worth noting that Option is not a subtype of Seq, as is often assumed.
An easy tip to remember, for comprehensions will try to return the type of the collection of the first generator, Option[Int] in this case. So, if you start with Some(1) you should expect a result of Option[T].
If you want a result of List type, you should start with a List generator.
Why have this restriction and not assume you'll always want some sort of sequence? You can have a situation where it makes sense to return Option. Maybe you have an Option[Int] that you want to combine with something to get a Option[List[Int]], say with the following function: (i:Int) => if (i > 0) List.range(0, i) else None; you could then write this and get None when things don't "make sense":
val f = (i:Int) => if (i > 0) Some(List.range(0, i)) else None
for (i <- Some(5); j <- f(i)) yield j
// returns: Option[List[Int]] = Some(List(0, 1, 2, 3, 4))
for (i <- None; j <- f(i)) yield j
// returns: Option[List[Int]] = None
for (i <- Some(-3); j <- f(i)) yield j
// returns: Option[List[Int]] = None
How for comprehensions are expanded in the general case are in fact a fairly general mechanism to combine an object of type M[T] with a function (T) => M[U] to get an object of type M[U]. In your example, M can be Option or List. In general it has to be the same type M. So you can't combine Option with List. For examples of other things that can be M, look at subclasses of this trait.
Why did combining List[T] with (T) => Option[T] work though when you started with the List? In this case the library use a more general type where it makes sense. So you can combine List with Traversable and there is an implicit conversion from Option to Traversable.
The bottom line is this: think about what type you want the expression to return and start with that type as the first generator. Wrap it in that type if necessary.
It probably has something to do with Option not being an Iterable. The implicit Option.option2Iterable will handle the case where compiler is expecting second to be an Iterable. I expect that the compiler magic is different depending on the type of the loop variable.
I always found this helpful:
scala> val foo: Option[Seq[Int]] = Some(Seq(1, 2, 3, 4, 5))
foo: Option[Seq[Int]] = Some(List(1, 2, 3, 4, 5))
scala> foo.flatten
<console>:13: error: Cannot prove that Seq[Int] <:< Option[B].
foo.flatten
^
scala> val bar: Seq[Seq[Int]] = Seq(Seq(1, 2, 3, 4, 5))
bar: Seq[Seq[Int]] = List(List(1, 2, 3, 4, 5))
scala> bar.flatten
res1: Seq[Int] = List(1, 2, 3, 4, 5)
scala> foo.toSeq.flatten
res2: Seq[Int] = List(1, 2, 3, 4, 5)
Since Scala 2.13 Option was made IterableOnce
sealed abstract class Option[+A] extends IterableOnce[A] with Product with Serializable
so the following for comprehension works without the use of option2Iterable implicit conversion
scala> for {
| a <- List(1)
| b <- Some(41)
| } yield (a + b)
val res35: List[Int] = List(42)
scala> List(1).flatMap
final override def flatMap[B](f: Int => scala.collection.IterableOnce[B]): List[B]
where we see List#flatMap takes a function to IterableOnce. Desugaring above for comprehension we get something like
List(1).flatMap(a => Some(41).map(b => a + b))
which show the absence of the implicit conversion.
However in Scala 2.12 and before Option was not a traversable/iterable entity
sealed abstract class Option[+A] extends Product with Serializable
so the above for comprehension would desugar to something like
List(1).flatMap(a => option2Iterable(Some(41)).map(b => a + b))(List.canBuildFrom[Int])
where we see the implicit conversion.
The reason it does not work the other way around where for comprehension begins with Option and then we try to chain a List
scala> for {
| a <- Option(1)
| b <- List(41)
| } yield (a + b)
b <- List(41)
^
On line 3: error: type mismatch;
found : List[Int]
required: Option[?]
scala> Option(1).flatMap
final def flatMap[B](f: Int => Option[B]): Option[B]
is because Option#flatMap takes a function to Option and converting a List to Option probably does not make sense because we would lose elements for Lists with more than one element.
As szeiger explains
I think the recent Option changes actually make the for comprehensions
use case easier to understand because you do not need an implicit
conversion anymore. Option can be used on the RHS of a flatMap of any
collection type because it is IterableOnce (but not the opposite
because the RHS of Option#flatMap requires Option).