What is the best way to compute a function on each element in a collection with an incremental shadowing of each element, like this simple example :
val v = IndexedSeq(1,2,3,4)
v.shadowMap{ e => e + 1}
shadow 1: (3,4,5)
shadow 2: (2,4,5)
shadow 3: (2,3,5)
shadow 4: (2,3,4)
I think first to patch or slice to make this, but perhaps there is a more better way to do that in pure functional style ?
Thanks
Sr.
def shadowMap[A,B](xs: Seq[A])(f: A => B) = {
val ys = xs map f
for (i <- ys.indices; (as, bs) = ys splitAt i) yield as ++ bs.tail
}
You can define it like this:
class ShadowMapSeq[A, Repr <: Seq[A]](seq: SeqLike[A, Repr]) {
def shadowMap[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): Iterator[That] = {
seq.indices.iterator.map { i =>
val b = bf(seq.asInstanceOf[Repr])
b.sizeHint(seq.size - 1)
b ++= (seq.take(i) ++ seq.drop(i + 1)).map(f)
b.result
}
}
}
implicit def shadowMapSeq[A, Repr <: Seq[A]](seq: SeqLike[A, Repr]) = new ShadowMapSeq(seq)
And then use it like this:
scala> val v = IndexedSeq(1, 2, 3, 4)
scala> val results = v.shadowMap(_ + 1)
scala> results foreach println
Vector(3, 4, 5)
Vector(2, 4, 5)
Vector(2, 3, 5)
Vector(2, 3, 4)
If by "pure functional style" you mean something like "without making reference to indices" (since you say you want to avoid patch and slice), you can do this pretty elegantly with zippers. For example, here's how to write it with Scalaz's zipper implementation (this is just a demonstration—if you want to wrap it up more nicely you can use the approach dhg gives in his answer):
import scalaz._, Scalaz._
List(1, 2, 3, 4).map(_ + 1).toZipper.map(
_.positions.map(p => (p.lefts.reverse ++ p.rights).toList).toStream
).flatten
In general you're probably better off going with dhg's solution in real code, but the zipper is a handy data structure to know about, and it's a good fit for this problem.
Related
I have this code:
def distinct(seq: Seq[Int]): Seq[Int] =
seq.fold(SortedSet[Int]()) ((acc, i) => acc + i)
I want to iterate over seq, delete duplicates (keep the first number) and keep order of the numbers. My idea was to use a SortedSet as an acc.
But I am getting:
Type mismatch:
Required: String
Found: Any
How to solve this? (I also don't know how to convert SortedSet to Seq in the final iteration as I want distinct to return seq)
p.s. without using standard seq distinct method
Online code
You shouldn't use fold if you try to accumulate something with different type than container (SortedSet != Int) in your case. Look at signature fold:
def fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1
it takes accumulator with type A1 and combiner function (A1, A1) => A1 which combines two A1 elements.
In your case is better to use foldLeft which takes accumulator with different type than container:
def foldLeft[B](z: B)(op: (B, A) => B): B
it accumulates some B value using seed z and combiner from B and A to B.
In your case I would like to use LinkedHashSet it keeps the order of added elements and remove duplicates, look:
import scala.collection.mutable
def distinct(seq: Seq[Int]): Seq[Int] = {
seq.foldLeft(mutable.LinkedHashSet.empty[Int])(_ + _).toSeq
}
distinct(Seq(7, 2, 4, 2, 3, 0)) // ArrayBuffer(7, 2, 4, 3, 0)
distinct(Seq(0, 0, 0, 0)) // ArrayBuffer(0)
distinct(Seq(1, 5, 2, 7)) // ArrayBuffer(1, 5, 2, 7)
and after folding just use toSeq
be careful, lambda _ + _ is just syntactic sugar for combiner:
(linkedSet, nextElement) => linkedSet + nextElement
I would just call distinct on your Seq. You can see in the source-code of SeqLike, that distinct will just traverse the Seq und skip already seen data:
def distinct: Repr = {
val b = newBuilder
val seen = mutable.HashSet[A]()
for (x <- this) {
if (!seen(x)) {
b += x
seen += x
}
}
b.result
}
In Functional Programming in Scala, the author asks to express FoldRight via FoldLeft. And then the author offers the following implementation:
def foldRightViaFoldLeftAuthor[A, B](l: List[A], z: B)(f: (A, B) => B): B = {
foldLeft(l, (b: B) => b)((g, a) => b => g(f(a, b)))(z)
}
There have been a couple of questions like this asking to explain the author's solution. And probably a lot of people are still struggling to understand it.
While I was thinking about the task I came up with a different implementation that seems much more readable and easier to grasp at least for me
def foldRightViaFoldLeftMy[A, B](l: List[A], z: B)(f: (A, B) => B): B = {
foldLeft(l, z)(((g: (A, B) => B) => (b: B, a: A) => g(a, b)) (f))
}
So I basically prepare a function that converts f(a,b) to f(b,a) and now I'm able to call foldLeft that is tail-recursive.
So my questions are:
Is there any reason to implement it in the way the author did?
Are there any drawbacks in my implementation in comparison to the author's?
You've written an implementation that has the same signature as foldRight, but it doesn't have the right semantics when the combination operation isn't commutative. To take one example, a right fold with the empty list as zero and cons as the combination operation should be identity:
scala> val input = List(1, 2, 3)
input: List[Int] = List(1, 2, 3)
scala> val f: (Int, List[Int]) => List[Int] = _ :: _
f: (Int, List[Int]) => List[Int] = $$Lambda$1912/991363637#5e9bf744
scala> foldRightViaFoldLeftAuthor(input, List.empty[Int])(f)
res0: List[Int] = List(1, 2, 3)
But your implementation reverses the list:
scala> foldRightViaFoldLeftMy(input, List.empty[Int])(f)
res1: List[Int] = List(3, 2, 1)
This is because you're still folding from left to right, even though you've switched the order of the combination function's arguments. I find the diagrams on the Wikipedia page about fold useful for visualizing the difference. In your implementation the applications happen like this:
scala> f(3, f(2, f(1, Nil)))
res2: List[Int] = List(3, 2, 1)
While in the book's implementation you have something like this:
((b3: List[Int]) =>
((b2: List[Int]) =>
((b1: List[Int]) => identity(f(1, b1)))(f(2, b2)))(f(3, b3)
)
)(Nil)
Which boils down to:
scala> f(1, f(2, f(3, Nil)))
res3: List[Int] = List(1, 2, 3)
So the answer to both of your questions is "yes", there is an important difference between your implementation and the book's.
Why does this construction cause a Type Mismatch error in Scala?
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
<console>:6: error: type mismatch;
found : List[(Int, Int)]
required: Option[?]
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
If I switch the Some with the List it compiles fine:
for (first <- List(1,2,3); second <- Some(1)) yield (first,second)
res41: List[(Int, Int)] = List((1,1), (2,1), (3,1))
This also works fine:
for (first <- Some(1); second <- Some(2)) yield (first,second)
For comprehensions are converted into calls to the map or flatMap method. For example this one:
for(x <- List(1) ; y <- List(1,2,3)) yield (x,y)
becomes that:
List(1).flatMap(x => List(1,2,3).map(y => (x,y)))
Therefore, the first loop value (in this case, List(1)) will receive the flatMap method call. Since flatMap on a List returns another List, the result of the for comprehension will of course be a List. (This was new to me: For comprehensions don't always result in streams, not even necessarily in Seqs.)
Now, take a look at how flatMap is declared in Option:
def flatMap [B] (f: (A) ⇒ Option[B]) : Option[B]
Keep this in mind. Let's see how the erroneous for comprehension (the one with Some(1)) gets converted to a sequence of map calls:
Some(1).flatMap(x => List(1,2,3).map(y => (x, y)))
Now, it's easy to see that the parameter of the flatMap call is something that returns a List, but not an Option, as required.
In order to fix the thing, you can do the following:
for(x <- Some(1).toSeq ; y <- List(1,2,3)) yield (x, y)
That compiles just fine. It is worth noting that Option is not a subtype of Seq, as is often assumed.
An easy tip to remember, for comprehensions will try to return the type of the collection of the first generator, Option[Int] in this case. So, if you start with Some(1) you should expect a result of Option[T].
If you want a result of List type, you should start with a List generator.
Why have this restriction and not assume you'll always want some sort of sequence? You can have a situation where it makes sense to return Option. Maybe you have an Option[Int] that you want to combine with something to get a Option[List[Int]], say with the following function: (i:Int) => if (i > 0) List.range(0, i) else None; you could then write this and get None when things don't "make sense":
val f = (i:Int) => if (i > 0) Some(List.range(0, i)) else None
for (i <- Some(5); j <- f(i)) yield j
// returns: Option[List[Int]] = Some(List(0, 1, 2, 3, 4))
for (i <- None; j <- f(i)) yield j
// returns: Option[List[Int]] = None
for (i <- Some(-3); j <- f(i)) yield j
// returns: Option[List[Int]] = None
How for comprehensions are expanded in the general case are in fact a fairly general mechanism to combine an object of type M[T] with a function (T) => M[U] to get an object of type M[U]. In your example, M can be Option or List. In general it has to be the same type M. So you can't combine Option with List. For examples of other things that can be M, look at subclasses of this trait.
Why did combining List[T] with (T) => Option[T] work though when you started with the List? In this case the library use a more general type where it makes sense. So you can combine List with Traversable and there is an implicit conversion from Option to Traversable.
The bottom line is this: think about what type you want the expression to return and start with that type as the first generator. Wrap it in that type if necessary.
It probably has something to do with Option not being an Iterable. The implicit Option.option2Iterable will handle the case where compiler is expecting second to be an Iterable. I expect that the compiler magic is different depending on the type of the loop variable.
I always found this helpful:
scala> val foo: Option[Seq[Int]] = Some(Seq(1, 2, 3, 4, 5))
foo: Option[Seq[Int]] = Some(List(1, 2, 3, 4, 5))
scala> foo.flatten
<console>:13: error: Cannot prove that Seq[Int] <:< Option[B].
foo.flatten
^
scala> val bar: Seq[Seq[Int]] = Seq(Seq(1, 2, 3, 4, 5))
bar: Seq[Seq[Int]] = List(List(1, 2, 3, 4, 5))
scala> bar.flatten
res1: Seq[Int] = List(1, 2, 3, 4, 5)
scala> foo.toSeq.flatten
res2: Seq[Int] = List(1, 2, 3, 4, 5)
Since Scala 2.13 Option was made IterableOnce
sealed abstract class Option[+A] extends IterableOnce[A] with Product with Serializable
so the following for comprehension works without the use of option2Iterable implicit conversion
scala> for {
| a <- List(1)
| b <- Some(41)
| } yield (a + b)
val res35: List[Int] = List(42)
scala> List(1).flatMap
final override def flatMap[B](f: Int => scala.collection.IterableOnce[B]): List[B]
where we see List#flatMap takes a function to IterableOnce. Desugaring above for comprehension we get something like
List(1).flatMap(a => Some(41).map(b => a + b))
which show the absence of the implicit conversion.
However in Scala 2.12 and before Option was not a traversable/iterable entity
sealed abstract class Option[+A] extends Product with Serializable
so the above for comprehension would desugar to something like
List(1).flatMap(a => option2Iterable(Some(41)).map(b => a + b))(List.canBuildFrom[Int])
where we see the implicit conversion.
The reason it does not work the other way around where for comprehension begins with Option and then we try to chain a List
scala> for {
| a <- Option(1)
| b <- List(41)
| } yield (a + b)
b <- List(41)
^
On line 3: error: type mismatch;
found : List[Int]
required: Option[?]
scala> Option(1).flatMap
final def flatMap[B](f: Int => Option[B]): Option[B]
is because Option#flatMap takes a function to Option and converting a List to Option probably does not make sense because we would lose elements for Lists with more than one element.
As szeiger explains
I think the recent Option changes actually make the for comprehensions
use case easier to understand because you do not need an implicit
conversion anymore. Option can be used on the RHS of a flatMap of any
collection type because it is IterableOnce (but not the opposite
because the RHS of Option#flatMap requires Option).
I'm trying to combine two Option[Iterable[_]] into a new Option[Iterable[_]]. I would like to return a Some if one (or both) of the elements is a Some and a None otherwise. It seems like there should be an idiomatic way of doing this, but I can't seem to find one. The following seems to do what I want, but isn't quite the slick solution I was hoping for.
def merge(
i1: Option[Iterable[_]], i2: Option[Iterable[_]]
): Option[Iterable[_]] = (i1, i2) match {
case (Some(as), Some(bs)) => Some(as ++ bs)
case (a # Some(as), None) => a
case (None, b # Some(bs)) => b
case _ => None
}
Any tips are appreciated. Thanks!
If you're willing to put up with a bit of abstract algebra, there's a nice generalization here: Iterable[_] is a monoid under concatenation, where a monoid's just a set of things (iterable collections, in this case) and an addition-like operation (concatenation) with some simple properties and an identity element (the empty collection).
Similarly, if A is a monoid, then Option[A] is also a monoid under a slightly more general version of your merge:
Some(xs) + Some(ys) == Some(xs + ys)
Some(xs) + None == Some(xs)
None + Some(ys) == Some(ys)
None + None == None
(Note that we need the fact that A is a monoid to know what to do in the first line.)
The Scalaz library captures all these generalizations in its Monoid type class, which lets you write your merge like this:
import scalaz._, Scalaz._
def merge(i1: Option[Iterable[_]], i2: Option[Iterable[_]]) = i1 |+| i2
Which works as expected:
scala> merge(Some(1 to 5), None)
res0: Option[Iterable[_]] = Some(Range(1, 2, 3, 4, 5))
scala> merge(Some(1 to 5), Some(4 :: 3 :: 2 :: 1 :: Nil))
res1: Option[Iterable[_]] = Some(Vector(1, 2, 3, 4, 5, 4, 3, 2, 1))
scala> merge(None, None)
res2: Option[Iterable[_]] = None
(Note that there are other operations that would give valid Monoid instances for Iterable and Option, but yours are the most commonly used, and the ones that Scalaz provides by default.)
This works:
def merge(i1: Option[Iterable[_]], i2: Option[Iterable[_]]): Option[Iterable[_]] =
(for (a <- i1; b <- i2) yield a ++ b).orElse(i1).orElse(i2)
The for/yield portion will add the contents of the options if and only if both are Some.
You can also drop some of the dots and parentheses if you want:
(for (a <- i1; b <- i2) yield a ++ b) orElse i1 orElse i2
You could use this for arbitrary arity:
def merge(xs: Option[Iterable[_]]*) =
if (xs.forall(_.isEmpty)) None else Some(xs.flatten.flatten)
Why does this construction cause a Type Mismatch error in Scala?
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
<console>:6: error: type mismatch;
found : List[(Int, Int)]
required: Option[?]
for (first <- Some(1); second <- List(1,2,3)) yield (first,second)
If I switch the Some with the List it compiles fine:
for (first <- List(1,2,3); second <- Some(1)) yield (first,second)
res41: List[(Int, Int)] = List((1,1), (2,1), (3,1))
This also works fine:
for (first <- Some(1); second <- Some(2)) yield (first,second)
For comprehensions are converted into calls to the map or flatMap method. For example this one:
for(x <- List(1) ; y <- List(1,2,3)) yield (x,y)
becomes that:
List(1).flatMap(x => List(1,2,3).map(y => (x,y)))
Therefore, the first loop value (in this case, List(1)) will receive the flatMap method call. Since flatMap on a List returns another List, the result of the for comprehension will of course be a List. (This was new to me: For comprehensions don't always result in streams, not even necessarily in Seqs.)
Now, take a look at how flatMap is declared in Option:
def flatMap [B] (f: (A) ⇒ Option[B]) : Option[B]
Keep this in mind. Let's see how the erroneous for comprehension (the one with Some(1)) gets converted to a sequence of map calls:
Some(1).flatMap(x => List(1,2,3).map(y => (x, y)))
Now, it's easy to see that the parameter of the flatMap call is something that returns a List, but not an Option, as required.
In order to fix the thing, you can do the following:
for(x <- Some(1).toSeq ; y <- List(1,2,3)) yield (x, y)
That compiles just fine. It is worth noting that Option is not a subtype of Seq, as is often assumed.
An easy tip to remember, for comprehensions will try to return the type of the collection of the first generator, Option[Int] in this case. So, if you start with Some(1) you should expect a result of Option[T].
If you want a result of List type, you should start with a List generator.
Why have this restriction and not assume you'll always want some sort of sequence? You can have a situation where it makes sense to return Option. Maybe you have an Option[Int] that you want to combine with something to get a Option[List[Int]], say with the following function: (i:Int) => if (i > 0) List.range(0, i) else None; you could then write this and get None when things don't "make sense":
val f = (i:Int) => if (i > 0) Some(List.range(0, i)) else None
for (i <- Some(5); j <- f(i)) yield j
// returns: Option[List[Int]] = Some(List(0, 1, 2, 3, 4))
for (i <- None; j <- f(i)) yield j
// returns: Option[List[Int]] = None
for (i <- Some(-3); j <- f(i)) yield j
// returns: Option[List[Int]] = None
How for comprehensions are expanded in the general case are in fact a fairly general mechanism to combine an object of type M[T] with a function (T) => M[U] to get an object of type M[U]. In your example, M can be Option or List. In general it has to be the same type M. So you can't combine Option with List. For examples of other things that can be M, look at subclasses of this trait.
Why did combining List[T] with (T) => Option[T] work though when you started with the List? In this case the library use a more general type where it makes sense. So you can combine List with Traversable and there is an implicit conversion from Option to Traversable.
The bottom line is this: think about what type you want the expression to return and start with that type as the first generator. Wrap it in that type if necessary.
It probably has something to do with Option not being an Iterable. The implicit Option.option2Iterable will handle the case where compiler is expecting second to be an Iterable. I expect that the compiler magic is different depending on the type of the loop variable.
I always found this helpful:
scala> val foo: Option[Seq[Int]] = Some(Seq(1, 2, 3, 4, 5))
foo: Option[Seq[Int]] = Some(List(1, 2, 3, 4, 5))
scala> foo.flatten
<console>:13: error: Cannot prove that Seq[Int] <:< Option[B].
foo.flatten
^
scala> val bar: Seq[Seq[Int]] = Seq(Seq(1, 2, 3, 4, 5))
bar: Seq[Seq[Int]] = List(List(1, 2, 3, 4, 5))
scala> bar.flatten
res1: Seq[Int] = List(1, 2, 3, 4, 5)
scala> foo.toSeq.flatten
res2: Seq[Int] = List(1, 2, 3, 4, 5)
Since Scala 2.13 Option was made IterableOnce
sealed abstract class Option[+A] extends IterableOnce[A] with Product with Serializable
so the following for comprehension works without the use of option2Iterable implicit conversion
scala> for {
| a <- List(1)
| b <- Some(41)
| } yield (a + b)
val res35: List[Int] = List(42)
scala> List(1).flatMap
final override def flatMap[B](f: Int => scala.collection.IterableOnce[B]): List[B]
where we see List#flatMap takes a function to IterableOnce. Desugaring above for comprehension we get something like
List(1).flatMap(a => Some(41).map(b => a + b))
which show the absence of the implicit conversion.
However in Scala 2.12 and before Option was not a traversable/iterable entity
sealed abstract class Option[+A] extends Product with Serializable
so the above for comprehension would desugar to something like
List(1).flatMap(a => option2Iterable(Some(41)).map(b => a + b))(List.canBuildFrom[Int])
where we see the implicit conversion.
The reason it does not work the other way around where for comprehension begins with Option and then we try to chain a List
scala> for {
| a <- Option(1)
| b <- List(41)
| } yield (a + b)
b <- List(41)
^
On line 3: error: type mismatch;
found : List[Int]
required: Option[?]
scala> Option(1).flatMap
final def flatMap[B](f: Int => Option[B]): Option[B]
is because Option#flatMap takes a function to Option and converting a List to Option probably does not make sense because we would lose elements for Lists with more than one element.
As szeiger explains
I think the recent Option changes actually make the for comprehensions
use case easier to understand because you do not need an implicit
conversion anymore. Option can be used on the RHS of a flatMap of any
collection type because it is IterableOnce (but not the opposite
because the RHS of Option#flatMap requires Option).