I want to write an unfold for streams.
Fails to compile, because it's not tail recursive.
#annotation.tailrec
def unfold[A, B](a: A)(f: A => Option[(A, B)]): Stream[B] =
f(a).map { case (a, b) => b #:: unfold(a)(f) }.getOrElse(Stream.empty)
... not really working either. (Not lazy)
def unfold[A, B](a: A)(f: A => Option[(A, B)]): Stream[B] = {
#annotation.tailrec
def go(state: A, next: Stream[B]): Stream[B] = {
f(state) match {
case Some((a, b)) => go(a, b #:: next)
case None => Stream.empty
}
}
go(a, Stream.empty)
}
// unfold: [A, B](a: A)(f: A => Option[(A, B)])Stream[B]
unfold(0)(i => Some((i+1, i))).take(10).toList
// java.lang.OutOfMemoryError: GC overhead limit exceeded
// at .$anonfun$res3$1(<console>:18)
// at .$anonfun$res3$1$adapted(<console>:18)
// at $$Lambda$1543/299219071.apply(Unknown Source)
// at .go$1(<console>:19)
// at .unfold(<console>:24)
// ... 27 elided
Kinda ugly, via Iterator.
def unfold[A, B](a: A)(f: A => Option[(A, B)]): Stream[B] = {
new Iterator[B] {
private var state: Option[A] = Some(a)
private var nextResult: Option[B] = None
def next: B =
nextResult.getOrElse(
throw new NoSuchElementException("next on empty iterator"))
def hasNext: Boolean = {
if (nextResult.isDefined) {
true
} else {
state match {
case Some(s) =>
f(s) match {
case Some((nextState, produced)) =>
nextResult = Some(produced)
state = Some(nextState)
true
case None => false
}
case None => false
}
}
}
}.toStream
}
def unfold[A, B](a: A)(f: A => Option[(A, B)]): Stream[B] =
f(a).map { case (a, b) => b #:: unfold(a)(f) }.getOrElse(Stream.empty)
This is correct and lazy (although some REPLs might attempt to print all the stream).
Scala streams have their tails evaluated lazily. I don't think you can affect the tail of a stream in a lazy fashion in a tail-recursive way. However, it's pointless because laziness gives you stack safety:
assert(unfold(0) { _ => Some((0, 0)) } .toString == "Stream(0, ?)")
assert {
try {
unfold(0) { _ => Some((0, 0)) }.take(1000000).toList
true
} catch {
case _: StackOverflowError => false
}
}
Related
Was playing with Lazy Structure Stream as below
import Stream._
sealed trait Stream[+A] {
..
def toList: List[A] = this match {
case Empty => Nil
case Cons(h, t) => println(s"${h()}::t().toList"); h()::t().toList
}
def foldRight[B](z: B) (f: ( A, => B) => B) : B = this match {
case Empty => println(s"foldRight of Empty return $z"); z
case Cons(h, t) => println(s"f(${h()}, t().foldRight(z)(f))"); f(h(), t().foldRight(z)(f))
}
..
}
case object Empty extends Stream[Nothing]
case class Cons[+A](h: () => A, t: () => Stream[A]) extends Stream[A]
object Stream {
def cons[A](h: => A, t: => Stream[A]): Stream[A] = {
lazy val hd = h
lazy val tl = t
Cons[A](() => hd, () => tl)
}
def empty[A]: Stream[A] = Empty
def apply[A](la: A*): Stream[A] = la match {
case list if list.isEmpty => empty[A]
case _ => cons(la.head, apply(la.tail:_*))
}
}
For a function takeWhile via foldRight i initially wrote:
def takeWhileFoldRight_0(p: A => Boolean) : Stream[A] = {
foldRight(empty[A]) {
case (a, b) if p(a) => println(s"takeWhileFoldRight cons($a, b) with p(a) returns: cons($a, b)"); cons(a, b)
case (a, b) if !p(a) => println(s"takeWhileFoldRight cons($a, b) with !p(a) returns: empty[A]"); empty[A]
}
}
Which when called as:
Stream(4,5,6).takeWhileFoldRight_0(_%2 == 0).toList
result in the following trace:
f(4, t().foldRight(z)(f))
f(5, t().foldRight(z)(f))
f(6, t().foldRight(z)(f))
foldRight of Empty return Empty
takeWhileFoldRight cons(6, b) with p(a) returns: cons(6, b)
takeWhileFoldRight cons(5, b) with !p(a) returns: empty[A]
takeWhileFoldRight cons(4, b) with p(a) returns: cons(4, b)
4::t().toList
res2: List[Int] = List(4)
Then questioning and questioning i figured that it might have been the unapply method in the pattern match that evaluate eagerly.
So i changed to
def takeWhileFoldRight(p: A => Boolean) : Stream[A] = {
foldRight(empty[A]) { (a, b) =>
if (p(a)) cons(a, b) else empty[A]
}
}
which when called as
Stream(4,5,6).takeWhileFoldRight(_%2 == 0).toList
result in the following trace:
f(4, t().foldRight(z)(f))
4::t().toList
f(5, t().foldRight(z)(f))
res1: List[Int] = List(4)
Hence my question:
Is there a way to recover the power of pattern match when working with by-name parameter ?
Said differently case i match parameter that are by-name without evaluating them eagerly ?
Or i have to go to a set of ugly nested "if" :p in that kind of scenario
Take a closer look at this fragment:
def toList: List[A] = this match {
case Empty => Nil
case Cons(h, t) => println(s"${h()}::t().toList"); h()::t().toList
}
def foldRight[B](z: B) (f: ( A, => B) => B) : B = this match {
case Empty => println(s"foldRight of Empty return $z"); z
case Cons(h, t) => println(s"f(${h()}, t().foldRight(z)(f))"); f(h(), t().foldRight(z)(f))
}
..
}
Here h and t in Cons aren't evaluated by unapply - after all unapply returns () => X functions without calling them. But you do. Twice for each match - once for printing and once for passing the result on. And you aren't remembering the result, so any future fold, map, etc would evaluate the function anew.
Depending on what behavior you want to have you should either:
Calculate the results once, right after matching them:
case Cons(h, t) =>
val hResult = h()
val tResult = t()
println(s"${hResult}::tail.toList")
hResult :: tResult.toList
or
not use case class because it cannot memoize the result and you might need to memoize it:
class Cons[A](fHead: () => A, fTail: () => Stream[A]) extends Stream[A] {
lazy val head: A = fHead()
lazy val tail: Stream[A] = fTail()
// also override: toString, equals, hashCode, ...
}
object Cons {
def apply[A](head: => A, tail: => Stream[A]): Stream[A] =
new Cons(() => head, () => tail)
def unapply[A](stream: Stream[A]): Option[(A, Stream[A])] = stream match {
case cons: Cons[A] => Some((cons.head, cons.tail)) // matches on type, doesn't use unapply
case _ => None
}
}
If you understand what you're doing you could also create a case class with overridden apply and unapply (like above) but that is almost always a signal that you shouldn't use a case class in the first place (because most likely toString, equals, hashCode, etc would have nonsensical implementation).
I'm working through the book Functional Programming in Scala, and at the end of the data structures chapter you are asked to implement the filter method in terms of flatMap. Here are the necessary functions and implementations:
sealed trait List[+A]
case object Nil extends List[Nothing]
case class Cons[+A](head: A, tail: List[A]) extends List[A]
object List {
def apply[A](as: A*): List[A] = {
if (as.isEmpty) Nil
else Cons(as.head, apply(as.tail: _*))
}
def append[A](l1: List[A], l2: List[A]): List[A] = {
foldRight(l1, l2)((elem, acc) => Cons(elem, acc))
}
def concat[A](ls: List[List[A]]): List[A] = {
foldLeft(ls, Nil: List[A])(append)
}
def map[A, B](l: List[A])(f: A => B): List[B] = {
foldRight(l, Nil: List[B])((elem, acc) => Cons(f(elem), acc))
}
def filter[A](l: List[A])(f: A => Boolean): List[A] = {
List.flatMap(l)(a => if (f(a)) List(a) else Nil)
}
def flatMap[A, B](l: List[A])(f: A => List[B]): List[B] = {
concat(map(l)(f))
}
def foldRight[A, B](l: List[A], z: B)(f: (A, B) => B): B = {
l match {
case Nil => z
case Cons(h, t) => f(h, foldRight(t, z)(f))
}
}
def foldLeft[A, B](l: List[A], z: B)(f: (B, A) => B): B = {
l match {
case Nil => z
case Cons(h, t) => foldLeft(t, f(z, h))(f)
}
}
}
The actual function call is here:
val x = List(1, 2, 3, 4, 5)
List.filter(x)(_ < 3)
As far as I can follow, after the map step you will have a List that looks like this:
Cons(Cons(1, Nil), Cons(2, Nil), Cons(Nil, Nil)...
I'm having trouble seeing where elements that are Nil are filtered out from the final result.
They are not "filtered out". They simply disappear after you apply concat on the list of lists, because concatenation with an empty list does nothing.
I have a MyStream-trait:
trait MyStream[+A] {
def uncons: Option[(A, MyStream[A])]
def isEmpty: Boolean = uncons.isEmpty
}
object MyStream {
def empty[A]: MyStream[A] =
new MyStream[A] { def uncons = None }
def cons[A](hd: => A, tl: => MyStream[A]): MyStream[A] =
new MyStream[A] { lazy val uncons = Some((hd, tl)) }
def apply[A](as: A*): MyStream[A] = if (as.isEmpty) empty
else cons(as.head, apply(as.tail: _*))
}
How can I use the uncons operator for pattern matching like:
def takeWhile(f: A => Boolean): MyStream[A] = this match {
case uncons(h,t) if f(h()) => cons(h(), t() takeWhile f)
case _ => empty
}
I am very new to Scala so I need a little help here.
Thanks!
How about this:
trait MyStream[+A] {
def uncons: Option[(A, MyStream[A])]
def isEmpty: Boolean = uncons.isEmpty
def takeWhile(f: A => Boolean): MyStream[A] = this match {
case MyStream(h, t) if f(h) => MyStream.cons(h, t takeWhile f)
case _ => MyStream.empty
}
#tailrec
final def foldLeft[B](z: B)(op: (B, A) => B): B =
this match {
case MyStream(h, t) => t.foldLeft(op(z, h))(op)
case _ => z
}
override def toString = this.foldLeft("") { case (acc, x) => acc + x }
}
object MyStream {
def empty[A]: MyStream[A] =
new MyStream[A] {
def uncons = None
}
def cons[A](hd: => A, tl: => MyStream[A]): MyStream[A] =
new MyStream[A] {
lazy val uncons = Some((hd, tl))
}
def apply[A](as: A*): MyStream[A] =
if (as.isEmpty) empty
else cons(as.head, apply(as.tail: _*))
def unapply[A](stream: MyStream[A]) = stream.uncons
}
object TestMyStream extends App {
import MyStream._
val s = cons(1, cons(2, cons(3, empty)))
println("All: " + s)
println("Take < 3: " + s.takeWhile(_ < 3))
}
prints:
All: 123
Take < 3: 12
I'd like a function that maps a function f over a sequence xs, and if f(x) (where x is an element of xs) produces a Failure then don't process any further elements of xs but immediately return Failure. If f(x) succeeds for all x then return a Success containing a sequence of the results.
So the type signature might be something like
def traverse[A, B](xs: Seq[A])(f: A => Try[B]): Try[Seq[B]]
And some test cases:
def doWork(i: Int): Try[Int] = {
i match {
case 1 => Success(10)
case 2 => Failure(new IllegalArgumentException("computer says no"))
case 3 => Success(30)
}
}
traverse(Seq(1,2,3))(doWork)
res0: scala.util.Try[Seq[Int]] = Failure(java.lang.IllegalArgumentException: computer says no)
traverse(Seq(1,3))(doWork)
scala.util.Try[Seq[Int]] = Success(List(10, 30))
What would be the most elegant way to implement this?
Simple implementation:
def traverse[A, B](xs: Seq[A])(f: A => Try[B]): Try[Seq[B]] =
xs.foldLeft[Try[Seq[B]]](Success(Vector())) { (attempt, elem) => for {
seq <- attempt
next <- f(elem)
} yield seq :+ next
}
Trouble here that while function will not evaluate f after the Failure will occur, function will traverse the sequence to the end , which could be undesirable in case of some complex Stream, so we may use some specialized version:
def traverse1[A, B](xs: Seq[A])(f: A => Try[B]): Try[Seq[B]] = {
val ys = xs map f
ys find (_.isFailure) match {
case None => Success(ys map (_.get))
case Some(Failure(ex)) => Failure(ex)
}
}
which uses intermediate collection, which leads to unnecessary memory overhead in case of strict collection
or we could reimplement fold from scratch:
def traverse[A, B](xs: Seq[A])(f: A => Try[B]): Try[Seq[B]] = {
def loop(xs: Seq[A], acc: Seq[B]): Try[Seq[B]] = xs match {
case Seq() => Success(acc)
case elem +: tail =>
f(elem) match {
case Failure(ex) => Failure(ex)
case Success(next) => loop(tail, acc :+ next)
}
}
loop(xs, Vector())
}
As we could see inner loop will continue iterations while it deals only with Success
One way, but is it the most elegant?
def traverse[A, B](xs: Seq[A])(f: A => Try[B]): Try[Seq[B]] = {
Try(xs.map(f(_).get))
}
Hello I'm reading the book called "Functional Programming in Scala". Book had code like this
trait Stream[+A] {
def uncons: Option[(A, Stream[A])]
def isEmpty: Boolean = uncons.isEmpty
}
object Stream {
...
def cons[A](hd: => A, tl: => Stream[A]): Stream[A] =
new Stream[A] {
lazy val uncons = Some((hd, tl))
}
...
it says that hd: => A syntax is making this a lazy val
But the code in the github repo had something like this
trait Stream[+A]{
def toList: List[A] = {
#annotation.tailrec
def go(s: Stream[A], acc: List[A]) : List[A] = s match {
case Cons(h,t) => go(t(), h() :: acc)
case _ => acc
}
go(this, List()).reverse
}
}
case class Cons[+A](h: ()=>A, t: () => Stream[A]) extends Stream[A]
What is the difference between h: =>A and h: ()=>A.
Why is the second part of code has () after argument like go(t(), h(), :: acc)