I have a function in my controller like this:
def getPreviousVersions(id: Int): Action[AnyContent] = Action.async {
val result: Future[Option[ElementModel]] = dto.getPreviousVersion(id)
// while-loop or whatever is best practice
// val arrayOfElements: Future[Seq[ElementModel] = ...
val c = for {
previousVersions <- arrayOfElements
} yield previousVersions
//do something with the versions
//Return something in the end
}
My model looks like this:
case class ElementModel(
...
previousVersion: Option[Int],
...)
I store the id of the latest previousVersion in my model. Now, what I want to do is iterate either recursively or with a while-loop or whatever is best-practice to get the previousVersion of the previousVersion and so on.
The idea is to get all previous versions, store them in a sequence and pass this seq to another function.
Is there a smooth, proper way to do this? Thanks!
def getVersionSequence(id: Int): Future[List[ElementModel]] = {
def _getVersionSequence(id: Int, fList: Future[List[ElementModel]]): Future[List[ElementModel]] = {
dto.getPreviousVersion(id).flatMap({
case Some(elementModel) => elementModel.previousVersion match {
case Some(pVId) => _getVersionSequence(pVId, fList.map(list => elementModel +: list))
case None => fList.map(list => elementModel +: list)
}
case None => fList
})
}
val fInvertedList = _getVersionSequence(id, Future(List.empty[ElementModel]))
fInvertedList.map(list => list.reverse)
}
def getPreviousVersions(id: Int): Action[AnyContent] = Action.async {
val c: Future[List[ElementModel]] = getVersionSequence(id)
//do something with the versions
//Return something in the end
}
def getPreviousVersion(previousVersionId: Int): Future[ElementModel] = dto.getElementModelForId(previousVersionId)
/*This has been changed from your question, you shouldn't need a getPreviousVersion(id) function in your database connector, but merely a function to get an element by id*/
def getAllPreviousVersions(e: ElementModel): Future[Seq[ElementModel]] = {
e.previousVersion match {
case None => Future.successful(Seq(e))
case Some(id) => getPreviousVersion(id).flatMap {
previousVersionElement =>
getAllPreviousVersions(previousVersionElement).map {
//Properly preserves order !
seq => previousVersionElement +: seq
}
}
}
}
def getPreviousVersions(e: ElementModel) = {
getAllPreviousVersions(e).map {
//do something with the versions
//Return something in the end
???
}
}
Related
I'm new to Scala, and I'm running into this strange situation.
def bar[A, B](implicit foo: A => B): B = {
// do something
foo
}
And then I got error like
require B but found A => B
How should I get B form A => B
Here's the reason why I did this, I have two functions:
def funcA: String = {
def getStrA: String = "A"
// then there's the same operation in both functions
Try{ } match {
case Success(_) => getStrA
case Failure(_) => // exactlly same error handler in both function
}
}
def funcB: Int = {
def doSomething(x: Int): Int = {
// do something
x / 1
}
val x = 1
Try{ } match {
case Success(_) => doSomething(1)
case Failure(_) => // exactlly same error handler in both function
}
}
Here's what I want to achieve
def funcA: String = {
implicit def getStrA: String = "A"
bar
}
def funcB: Int = {
val x = 1
implicit def doSomething(x: Int): Int = {
// do something
x / 1
}
bar
}
def bar[A, B](implicit foo: A => B): B = {
Try{ } match {
case Success(_) => foo
case Failure(_) => // exactlly same error handler in both function
}
}
You have a conversion from A to B. You need to return B. The only way to do this is to pass A into the function. This signature has an implied assumption that you have some valid A value (most likely hardcoded) that you will always use here.
def bar[A, B](implicit foo: A => B): B = {
val a: A = ... // hmm...
foo(a)
}
Considering, that A is parametric, then you are either missing some information, or this A is impossible to create (it cannot be null because not all types can take null as a value), so you might need to throw exception in such case. Probably you are either missing some A provider or you should always fail this operation.
UPDATE:
There is no need for using implicits at all in your code:
def bar[B](f: onSuccess: A => B) =
Try{ some operations } match {
case Success(value) => onSuccess(value)
case Failure(_) => // error handler
}
def funcA = bar(_ => "A")
def funcB = bar(_ => 1)
I am using a library that provides a Traversable[T] that pages through database results. I'd like to avoid loading the whole thing into memory, so I am trying to convert it to a Stream[T].
From what I can tell, the built in "asStream" method loads the whole Traversable into a Buffer, which defeats my purpose. My attempt (below) hits a StackOverflowException on large results, and I can't tell why. Can someone help me understand what is going on? Thanks!
def asStream[T](traversable: => Traversable[T]): Stream[T] = {
if (traversable.isEmpty) Empty
else {
lazy val head = traversable.head
lazy val tail = asStream(traversable.tail)
head #:: tail
}
}
Here's a complete example that reproduces this, based on a suggestion by #SCouto
import scala.collection.immutable.Stream.Empty
object StreamTest {
def main(args: Array[String]) = {
val bigVector = Vector.fill(90000)(1)
val optionStream = asStream(bigVector).map(v => Some(v))
val zipped = optionStream.zipAll(optionStream.tail, None, None)
}
def asStream[T](traversable: => Traversable[T]): Stream[T] = {
#annotation.tailrec
def loop(processed: => Stream[T], pending: => Traversable[T]): Stream[T] = {
if (pending.isEmpty) processed
else {
lazy val head = pending.head
lazy val tail = pending.tail
loop(processed :+ head, tail)
}
}
loop(Empty, traversable)
}
}
Edit: After some interesting ideas from #SCouto, I learned this could also be done with trampolines to keep the result as a Stream[T] that is in the original order
object StreamTest {
def main(args: Array[String]) = {
val bigVector = Range(1, 90000).toVector
val optionStream = asStream(bigVector).map(v => Some(v))
val zipped = optionStream.zipAll(optionStream.tail, None, None)
zipped.take(10).foreach(println)
}
def asStream[T](traversable: => Traversable[T]): Stream[T] = {
sealed trait Traversal[+R]
case class More[+R](result: R, next: () => Traversal[R]) extends Traversal[R]
case object Done extends Traversal[Nothing]
def next(currentTraversable: Traversable[T]): Traversal[T] = {
if (currentTraversable.isEmpty) Done
else More(currentTraversable.head, () => next(currentTraversable.tail))
}
def trampoline[R](body: => Traversal[R]): Stream[R] = {
def loop(thunk: () => Traversal[R]): Stream[R] = {
thunk.apply match {
case More(result, next) => Stream.cons(result, loop(next))
case Done => Stream.empty
}
}
loop(() => body)
}
trampoline(next(traversable))
}
}
Try this:
def asStream[T](traversable: => Traversable[T]): Stream[T] = {
#annotation.tailrec
def loop(processed: Stream[T], pending: Traversable[T]): Stream[T] = {
if (pending.isEmpty) processed
else {
lazy val head = pending.head
lazy val tail = pending.tail
loop(head #:: processed, tail)
}
}
loop(Empty, traversable)
}
The main point is to ensure that your recursive call is the last action of your recursive function.
To ensure this you can use both a nested method (called loop in the example) and the tailrec annotation which ensures your method is tail-safe.
You can find info about tail rec here and in this awesome answer here
EDIT
The problem was that we were adding the element at the end of the Stream. If you add it as head of the Stream as in your example it will work fine. I updated my code. Please test it and let us know the result.
My tests:
scala> val optionStream = asStream(Vector.fill(90000)(1)).map(v => Some(v))
optionStream: scala.collection.immutable.Stream[Some[Int]] = Stream(Some(1), ?)
scala> val zipped = optionStream.zipAll(optionStream.tail, None, None)
zipped: scala.collection.immutable.Stream[(Option[Int], Option[Int])] = Stream((Some(1),Some(1)), ?)
EDIT2:
According to your comments, and considering the fpinscala example as you said. I think this may help you. The point is creating a case class structure with lazy evaluation. Where the head is a single element, and the tail a traversable
sealed trait myStream[+T] {
def head: Option[T] = this match {
case MyEmpty => None
case MyCons(h, _) => Some(h())
}
def tail: myStream[T] = this match {
case MyEmpty => MyEmpty
case MyCons(_, t) => myStream.cons(t().head, t().tail)
}
}
case object MyEmpty extends myStream[Nothing]
case class MyCons[+T](h: () => T, t: () => Traversable[T]) extends myStream[T]
object myStream {
def cons[T](hd: => T, tl: => Traversable[T]): myStream[T] = {
lazy val head = hd
lazy val tail = tl
MyCons(() => head, () => tail)
}
def empty[T]: myStream[T] = MyEmpty
def apply[T](as: T*): myStream[T] = {
if (as.isEmpty) empty
else cons(as.head, as.tail)
}
}
Some Quick tests:
val bigVector = Vector.fill(90000)(1)
myStream.cons(bigVector.head, bigVector.tail)
res2: myStream[Int] = MyCons(<function0>,<function0>)
Retrieving head:
res2.head
res3: Option[Int] = Some(1)
And the tail:
res2.tail
res4: myStream[Int] = MyCons(<function0>,<function0>)
EDIT3
The trampoline solution by the op:
def asStream[T](traversable: => Traversable[T]): Stream[T] = {
sealed trait Traversal[+R]
case class More[+R](result: R, next: () => Traversal[R]) extends Traversal[R]
case object Done extends Traversal[Nothing]
def next(currentTraversable: Traversable[T]): Traversal[T] = {
if (currentTraversable.isEmpty) Done
else More(currentTraversable.head, () => next(currentTraversable.tail))
}
def trampoline[R](body: => Traversal[R]): Stream[R] = {
def loop(thunk: () => Traversal[R]): Stream[R] = {
thunk.apply match {
case More(result, next) => Stream.cons(result, loop(next))
case Done => Stream.empty
}
}
loop(() => body)
}
trampoline(next(traversable))
}
}
Stream doesn't keep the data in memory because you declare how to generate each item. It's very likely that your database data is not been procedurally generated so what you need is to fetch the data the first time you ask for it (something like def getData(index: Int): Future[Data]).
The biggest problem rise in, since you are fetching data from a database, you are probably using Futures so, even if you are able to achieve it, you would have a Future[Stream[Data]] object which is not that nice to use or, much worst, block it.
Wouldn't be much more worthy just to paginate your database data query?
I have an array of objects of type Either[A, B]. If I know for a particular element whether it is an A or a B, how do I call a method on it that only exists on one of the 2 types. For example:
import scala.util.Random
object EitherTest extends App {
def newObj(x: Int): Either[A,B] = {
if (x == 0)
Left(new A())
else
Right(new B())
}
val random = new Random()
val randomArray = (0 until 10).map(_ => random.nextInt(2))
val eitherArray = randomArray.map(newObj)
(0 until 10).foreach(x => randomArray(x) match {
case 0 => eitherArray(x).aMethod()
case 1 => eitherArray(x).bMethod()
case _ => println("Error!")
})
}
class A {
def aMethod() = println("A")
}
class B {
def bMethod() = println("B")
}
When I compile this code, the lines
case 0 => eitherArray(x).aMethod()
case 1 => eitherArray(x).bMethod()
both have the error "value aMethod is not a member of Either[A,B]". How can I solve this?
I don't know why fold doesn't get the respect it deserves. It can be so useful.
eitherArray.foreach(_.fold(_.aMethod(), _.bMethod()))
Well, you can do it if you exctract the logic to another method, and do some pattern matching over the value Either, then check if it is Right or Left, and that's it!
object HelloWorld {
import scala.util.Random
def main(args: Array[String]) {
val random = new Random()
val randomArray = (0 until 10).map(_ => random.nextInt(2))
val eitherArray = randomArray.map(EitherTest.newObj)
(0 until 10).foreach(x => randomArray(x) match {
case 0 => EitherTest.callmethod(eitherArray(x))
case 1 => EitherTest.callmethod(eitherArray(x))
case _ => println("Error!")
})
println("Hello, world!")
}
}
class EitherTest
object EitherTest {
def callmethod(ei : Either[A,B]) = {
ei match {
case Left(a) => a.aMethod()
case Right(b) => b.bMethod()
}
}
def newObj(x: Int): Either[A,B] = {
if (x == 0)
Left(new A())
else
Right(new B())
}
}
class A {
def aMethod() = println("A")
}
class B {
def bMethod() = println("B")
}
Will print for you, for one random example:
A
B
A
B
A
A
A
B
B
B
Hello, world!
Basically, the way you do with Either is projections: Either.left gives you the projection of the left type, and Either.right gives you that of the right.
The projections are somewhat similar to options, in that they can be empty (if your Either is a Right, then the left projection is empty and vice versa), and you can use the usual monadic transformations with them, like map, flatMap, foreach, getOrElse etc.
Your example, could look like this:
randomArray.foreach { either =>
either.left.foreach(_.aMethod)
either.right.foreach(_.bMethod)
}
You could also use pattern-matching instead, that's less general, but, perhaps looks a bit clearer in this case:
randomArray.foreach {
case Left(a) => a.aMethod
case Right(b) => b.bMethod
}
I have an Iterator[Record] which is ordered on record.id this way:
record.id=1
record.id=1
...
record.id=1
record.id=2
record.id=2
..
record.id=2
Records of a specific ID could occur a large number of times, so I want to write a function that takes this iterator as input, and returns an Iterator[Iterator[Record]] output in a lazy manner.
I was able to come up with the following, but it fails on StackOverflowError after 500K records or so:
def groupByIter[T, B](iterO: Iterator[T])(func: T => B): Iterator[Iterator[T]] = new Iterator[Iterator[T]] {
var iter = iterO
def hasNext = iter.hasNext
def next() = {
val first = iter.next()
val firstValue = func(first)
val (i1, i2) = iter.span(el => func(el) == firstValue)
iter = i2
Iterator(first) ++ i1
}
}
What am I doing wrong?
Trouble here is that each Iterator.span call makes another stacked closure for trailing iterator, and without any trampolining it's very easy to overflow.
Actually I dont think there is an implementation, which is not memoizing elements of prefix iterator, since followed iterator could be accessed earlier than prefix is drain out.
Even in .span implementation there is a Queue to memoize elements in the Leading definition.
So easiest implementation that I could imagine is the following via Stream.
implicit class StreamChopOps[T](xs: Stream[T]) {
def chopBy[U](f: T => U): Stream[Stream[T]] = xs match {
case x #:: _ =>
def eq(e: T) = f(e) == f(x)
xs.takeWhile(eq) #:: xs.dropWhile(eq).chopBy(f)
case _ => Stream.empty
}
}
Although it could be not the most performant as it memoize a lot. But with proper iterating of that, GC should handle problem of excess intermediate streams.
You could use it as myIterator.toStream.chopBy(f)
Simple check validates that following code can run without SO
Iterator.fill(10000000)(Iterator(1,1,2)).flatten //1,1,2,1,1,2,...
.toStream.chopBy(identity) //(1,1),(2),(1,1),(2),...
.map(xs => xs.sum * xs.size).sum //60000000
Inspired by chopBy implemented by #Odomontois here is a chopBy I implemented for Iterator. Of course each bulk should fit allocated memory. It doesn't looks very elegant but it seems to work :)
implicit class IteratorChopOps[A](toChopIter: Iterator[A]) {
def chopBy[U](f: A => U) = new Iterator[Traversable[A]] {
var next_el: Option[A] = None
#tailrec
private def accum(acc: List[A]): List[A] = {
next_el = None
val new_acc = hasNext match {
case true =>
val next = toChopIter.next()
acc match {
case Nil =>
acc :+ next
case _ MatchTail t if (f(t) == f(next)) =>
acc :+ next
case _ =>
next_el = Some(next)
acc
}
case false =>
next_el = None
return acc
}
next_el match{
case Some(_) =>
new_acc
case None => accum(new_acc)
}
}
def hasNext = {
toChopIter.hasNext || next_el.isDefined
}
def next: Traversable[A] = accum(next_el.toList)
}
}
And here is an extractor for matching tail:
object MatchTail {
def unapply[A] (l: Traversable[A]) = Some( (l.init, l.last) )
}
Is there a way to turn a Seq[Future[X]] into an Enumerator[X] ? The use case is that I want to get resources by crawling the web. This is going to return a Sequence of Futures, and I'd like to return an Enumerator that will push the futures in the order in which they are first finished on to the Iteratee.
It looks like Victor Klang's Future select gist could be used to do this - though it looks pretty inefficient.
Note: The Iteratees and Enumerator's in question are those given by the play framework version 2.x, ie with the following imports: import play.api.libs.iteratee._
Using Victor Klang's select method:
/**
* "Select" off the first future to be satisfied. Return this as a
* result, with the remainder of the Futures as a sequence.
*
* #param fs a scala.collection.Seq
*/
def select[A](fs: Seq[Future[A]])(implicit ec: ExecutionContext):
Future[(Try[A], Seq[Future[A]])] = {
#scala.annotation.tailrec
def stripe(p: Promise[(Try[A], Seq[Future[A]])],
heads: Seq[Future[A]],
elem: Future[A],
tail: Seq[Future[A]]): Future[(Try[A], Seq[Future[A]])] = {
elem onComplete { res => if (!p.isCompleted) p.trySuccess((res, heads ++ tail)) }
if (tail.isEmpty) p.future
else stripe(p, heads :+ elem, tail.head, tail.tail)
}
if (fs.isEmpty) Future.failed(new IllegalArgumentException("empty future list!"))
else stripe(Promise(), fs.genericBuilder[Future[A]].result, fs.head, fs.tail)
}
}
I can then get what I need with
Enumerator.unfoldM(initialSeqOfFutureAs){ seqOfFutureAs =>
if (seqOfFutureAs.isEmpty) {
Future(None)
} else {
FutureUtil.select(seqOfFutureAs).map {
case (t, seqFuture) => t.toOption.map {
a => (seqFuture, a)
}
}
}
}
A better, shorter and I think more efficient answer is:
def toEnumerator(seqFutureX: Seq[Future[X]]) = new Enumerator[X] {
def apply[A](i: Iteratee[X, A]): Future[Iteratee[X, A]] = {
Future.sequence(seqFutureX).flatMap { seqX: Seq[X] =>
seqX.foldLeft(Future.successful(i)) {
case (i, x) => i.flatMap(_.feed(Input.El(x)))
}
}
}
}
I do realise that the question is a bit old already, but based on Santhosh's answer and the built-in Enumterator.enumerate() implementation I came up with the following:
def enumerateM[E](traversable: TraversableOnce[Future[E]])(implicit ec: ExecutionContext): Enumerator[E] = {
val it = traversable.toIterator
Enumerator.generateM {
if (it.hasNext) {
val next: Future[E] = it.next()
next map {
e => Some(e)
}
} else {
Future.successful[Option[E]] {
None
}
}
}
}
Note that unlike the first Viktor-select-based-solution this one preserves the order, but you can still start off all computations asynchronously before. So, for example, you can do the following:
// For lack of a better name
def mapEachM[E, NE](eventuallyList: Future[List[E]])(f: E => Future[NE])(implicit ec: ExecutionContext): Enumerator[NE] =
Enumerator.flatten(
eventuallyList map { list =>
enumerateM(list map f)
}
)
This latter method was in fact what I was looking for when I stumbled on this thread. Hope it helps someone! :)
You could construct one using the Java Executor Completeion Service (JavaDoc). The idea is to use create a sequence of new futures, each using ExecutorCompletionService.take() to wait for the next result. Each future will start, when the previous future has its result.
But please b e aware, that this might be not that efficient, because a lot of synchronisation is happening behind the scenes. It might be more efficient, to use some parallel map reduce for calculation (e.g. using Scala's ParSeq) and let the Enumerator wait for the complete result.
WARNING: Not compiled before answering
What about something like this:
def toEnumerator(seqFutureX: Seq[Future[X]]) = new Enumerator[X] {
def apply[A](i: Iteratee[X, A]): Future[Iteratee[X, A]] =
Future.fold(seqFutureX)(i){ case (i, x) => i.flatMap(_.feed(Input.El(x)))) }
}
Here is something I found handy,
def unfold[A,B](xs:Seq[A])(proc:A => Future[B])(implicit errorHandler:Throwable => B):Enumerator[B] = {
Enumerator.unfoldM (xs) { xs =>
if (xs.isEmpty) Future(None)
else proc(xs.head) map (b => Some(xs.tail,b)) recover {
case e => Some((xs.tail,errorHandler(e)))
}
}
}
def unfold[A,B](fxs:Future[Seq[A]])(proc:A => Future[B]) (implicit errorHandler1:Throwable => Seq[A], errorHandler:Throwable => B) :Enumerator[B] = {
(unfold(Seq(fxs))(fxs => fxs)(errorHandler1)).flatMap(unfold(_)(proc)(errorHandler))
}
def unfoldFutures[A,B](xsfxs:Seq[Future[Seq[A]]])(proc:A => Future[B]) (implicit errorHandler1:Throwable => Seq[A], errorHandler:Throwable => B) :Enumerator[B] = {
xsfxs.map(unfold(_)(proc)).reduceLeft((a,b) => a.andThen(b))
}
I would like to propose the use of a Broadcast
def seqToEnumerator[A](futuresA: Seq[Future[A]])(defaultValue: A, errorHandler: Throwable => A): Enumerator[A] ={
val (enumerator, channel) = Concurrent.broadcast[A]
futuresA.foreach(f => f.onComplete({
case Success(Some(a: A)) => channel.push(a)
case Success(None) => channel.push(defaultValue)
case Failure(exception) => channel.push(errorHandler(exception))
}))
enumerator
}
I added errorHandling and defaultValues but you can skip those by using onSuccess or onFailure, instead of onComplete