I have a stateful algorithm that incrementally takes input and incrementally produces output. The inputs and outputs are unrelated in number; i.e. an input may produce zero or more outputs.
I am attempting to turn it into an Enumeratee in the Play Framework, but I am having difficulty getting started.
My algorithm has local mutable state and synchronous operations and looks something like this
var state = 'foo
var i = input()
while (i != null) {
if (state == 'foo && i >= 0) {
// 2 outputs, and change state
output(i - 2)
output(i * 3)
state = 'bar
} else if (state == 'foo) {
// error
error("expected >= 0")
} else if (state == 'bar) {
// 0 outputs, and change state
state = 'foo
}
... // etc
i = input()
}
if (state != 'bar) {
error("unexpected end")
}
I've studied the map, filter, etc. implementations in Enumeratee.scala, and I sort of understand them. But I'm having trouble seeing how to write my own implementation of something more complicated.
Could you describe/demonstrate how I can transform this algorithm into an Enumeratee?
The only thing you need to implement is the applyOn method:
def applyOn[A](inner: Iteratee[To, A]): Iteratee[From, Iteratee[To, A]] = ...
Everything else is implemented in the trait.
When creating an iteratee, I find recursion is the most important trick; it's a continuation-like style where rather than returning anything, each step computes the thing it needs to compute and then calls into it again. So your state should become a function parameter:
def next[A](inner: Iteratee[To, A], i: Input[From], state: Symbol)
: Iteratee[From, A] =
i match {
case Input.El(e) =>
if(state == 'foo && e >= 0) {
val nextInner = Iteratee.flatten {
inner.feed(Input.El(e - 2)) flatMap {_.feed(Input.El(e * 3))}
}
val nextState = 'bar
Cont {k => next(nextInner, k, nextState)}
} else if(state == 'foo)
Error("expected >=0", i)
else if(state == 'bar)
next(inner, i, 'foo)
...
case _ =>
//pass through Empty or Eof
Iteratee.flatten(inner.feed(i))
}
def applyOn[A](inner: Iteratee[To, A]): Iteratee[From, Iteratee[To, A]] =
Cont {i => next(inner, i, 'foo)}
Notice how we return either a direct recursive call to next, or a continuation that will eventually make a (mutually) recursive call to next.
I've passed the Input around explicitly to every call, whereas a more elegant approach might handle it in the continuation (and perhaps implement applyOn directly rather than having it as a wrapper method), but hopefully this makes it clear what's going on. I'm sure there are more elegant ways to achieve the desired result (I've used scalaz iteratees but I don't know the Play API at all), but it's nice to work through the explicit solution at least once so we understand what's really going on underneath.
Related
I have two objects of same type but I want to call methods using the objects but depending upon couple of boolean values.
def myMethod(a: Boolean, b: Boolean, obj1: MyClass, obj2: MyClass): Future[Done] = {
if(a) obj1.methodWhichReturnsFuture()
if(b) obj2.methodWhichReturnsFuture()
}
Here the method #methodWhichReturnsFuture has return typr Future[Done]. THis is showing error since I am returning Unit here. I though of creating a 'var' and assigning it inside both method call but looks like its not a good way. This is definitely basic but I want to know what is the better way to implement this? Thanks
In Scala, an if without an else can only be executed for side-effect, so the resulting value of
if (a) obj1.methodWhichReturnsFuture()
will be discarded and the singleton value for Unit (()) will be the result. The same will happen in the if (b) statement (and technically that () will be the result of myMethod, but it's the same () as from the if (a)).
You will thus need to add else clauses which also result in a Future[Done] and determine what you want the result to be if both a and b are true (furthermore, if you want both the obj1 and obj2 calls to be made, you will need to decide whether to execute them sequentially or in parallel) as well as if both a and b are false.
Thankfully, because Done is a singleton (it's basically the same as Unit as far as encoding "this happened", but in a way which is nicer for Java code using Akka), there's a reasonable default Future[Done]: Future.successful(Done) (which can basically be interpreted as "yeah, sure, I did it (but there was nothing to actually do)").
So if you want to do both the obj1 and obj2 calls simultaneously (or at least without a defined "happens-before" relationship), you could write:
val obj1Fut = if (a) obj1.methodWhichReturnsFuture() else Future.successful(Done)
val obj2Fut = if (b) obj2.methodWhichReturnsFuture() else Future.successful(Done)
// exploits eagerness of Future, won't complete until both obj1Fut and obj2Fut have completed
obj1Fut.flatMap(_ => obj2Fut)
If you don't want to perform the obj2 call until after the obj1 call:
val obj1Fut = if (a) obj1.methodWhichReturnsFuture() else Future.successful(Done)
obj1Fut.flatMap { _ =>
if (b) obj2.methodWhichReturnsFuture()
else Future.successful(Done)
}
Note that in the Unit returning code you presented, it's quite possible for the obj2 future to complete before obj1's future has completed.
And if you only wanted to do the obj1 call if both a and b were true:
if (a) obj1.methodWhichReturnsFuture()
else if (b) obj2.methodWhichReturnsFuture()
else Future.successful(Done)
// or the equivalent pattern match, which might be more legible than if else if else
// (a, b) match {
// case (true, _) => obj1.methodWhichReturnsFuture()
// case (false, true) => obj2.methodWhichReturnsFuture()
// case (false, false) => Future.successful(Done)
// }
If I replace the first line of the following recursive depth first search function with the lines commented out within the foreach block, it will fail to compile as a tail recursive function (due to the #tailrec annotation) even though the recursion is still clearly the last action of the function. Is there a legitimate reason for this behavior?
#tailrec def searchNodes(nodes: List[Node], visitedNodes: List[Node], end: String, currentLevel: Int) : Int = {
if (nodes.exists(n => n.id == end)) return currentLevel
val newVisitedNodes = visitedNodes ::: nodes
var nextNodes = List[Node]()
nodes.foreach(n => {
/*
if (n.id == end){
return currentLevel
}
*/
nextNodes = nextNodes ::: n.addAdjacentNodes(visitedNodes)
})
if (nextNodes.size == 0) return -1
return searchNodes(nextNodes, newVisitedNodes, end, currentLevel + 1)
}
As the other answer explains, using return in scala is a bad idea, and an anti-pattern. But what is even worse is using a return inside a lambda function (like your commented out code inside foreach): that actually throws an exception, that is then caught outside to make the main function exit.
As a result, the body of your function is compiled into something like:
def foo(nodes: List[Node]) = {
val handle = new AnyRef
try {
nodes.foreach { n =>
if(n.id == "foo") throw new NonLocalReturnControl(handle, currentLevel)
...
foo(nextNodes)
} catch {
case nlrc: NonLocalReturnControl[Int] if nlrc.key == handle => nlrc.value
}
}
As you can see, your recursive call is not in a tail position here, so compiler error is legit.
A more idiomatic way to write what you want would be to deconstruct the list, and use the recursion itself as the "engine" for the loop:
def searchNodes(nodes: List[Node], end: String) = {
#tailrec def doSearch(
nodes: List[(Node, Int)],
visited: List[Node],
end: String
) : Int = nodes match {
case Nil => -1
case (node, level) :: tail if node.id == end => level
case (node, level) :: tail =>
doSearch(
tail ::: node.addAdjacentNodes(visited).map(_ -> level+1),
node :: visited,
end
)
}
doSearch(nodes.map(_ -> 0), Nil, end)
}
I'm not sure exactly what the compiler is thinking, but I think all your return statements will be causing problems.
Using return is an antipattern in scala - you don't need to write it, and you shouldn't. To avoid it, you'll have to restructure your if ... return blocks as if ... value ... else ... other value blocks.
This shape is possible because everything is an expression (sort of). Your def has a value, which is defined by an if ... else block, where the if and the else both have values, and so on all the way down. If you want to ignore the value of something you can just put a new line after it, and the return value of a block is always the value of the last expression in it. You can do that to avoid having to rewrite your foreach, but it would be better to write it functionally, as a map.
I am migrating code from a synchronous to an async style. The problem is fairly simple: call a series of functions, and stop with the first one to return a non-error result (returning that value, otherwise the last computed value). I started with:
def find(fs: Seq[Function0[Int]], result: Int = -1): Int = {
if (fs.isEmpty) result
else {
val res = fs.head()
if (res != 0) res
else find(fs.tail, res)
}
}
However, if the functions become async, (i.e. return Future[Int]) I can't get the right invocation. For example,
def ffind(ffs: Seq[Function0[Future[Int]]], result: Future[Int] = Future { -1 }): Future[Int] = {
if (ffs.isEmpty) result
else ffind(ffs.tail, ffs.head())
}
works great, but it evaluates all the functions, regardless of return value. But something like:
def findBad(ffs: Seq[Function0[Future[Int]]], result: Future[Int] = Future { -1 }): Future[Int] = {
if (ffs.isEmpty) result
else {
ffs.head() map { res =>
if (res != 0) res
else findBad(ffs.tail, Future(res))
}
}
}
doesn't type-check. Any suggestions? We can suppose that each call of a function is expensive, so none should be called twice, nor any after the first 'successful' call in the sequence. TIA
Here's why it doesn't type-check: findBad returns a Future[Int], but mapping res into an invocation of findBad would result in a Future[Future[Int]]. You need to change map into flatMap. Note that now you also need to wrap res from the first condition (if res != 0) into a Future too, so that both branches return a Future. Here's the code:
ffs.head() flatMap { res =>
if (res != 0) Future.succesful(res)
else findBad(ffs.tail, Future(res))
}
BTW if you want to run them all and return whichever completes first, disregarding all the rest, then it's a bit different problem (see here), but you say that each function call is expensive so I doubt that's what you are trying to do.
Say there is a function with two optional parameters represent two different condition to filter.
def judge(a: Option[Int], b: Option[Int]) {
// here filter according the values of `a` and `b` if are set (not empty)
// if `a` or `b` is empty, then ignore.
// And if `a` and `b` both are empty, then filtering get nothing.
val objs: List[SomeObject] = ...
if (!(a.isEmpty) || !(b.isEmpty)) {
objs.filter { obj =>
a.map(_ == obj.a).getOrElse(true) && b.map(_ == obj.b).getOrElse(true)
}
} else {
List[SomeObject]()
}
}
It seems works, but I think the way is a little verbose. Is there another simpler way to do it ?
I would make two changes.
1. !(a.isEmpty) || !(b.isEmpty) is difficult to read and requires you (or at least me) to scrutinize the logic a little more to verify what it's actually doing.
(a.nonEmpty || b.nonEmpty) would be more expressive and logically equivalent.
2. I think it would be more idiomatic to use forall within the filter.
For example:
a.forall(_ == obj.a)
would be the same as:
a.map(_ == obj.a).getOrElse(true)
And I think it's more clear as to what it does. It's saying for all elements contained in a, they must be equal to obj.a. If a is empty, then by default all of it's elements are equal to obj.a (because it doesn't have any, so we can say anything about them and it will be true).
Now your function could look more like this:
def judge(a: Option[Int], b: Option[Int]): List[SomeObject] = {
val objs = ...
if (a.nonEmpty || b.nonEmpty) {
objs.filter { obj =>
a.forall(_ == obj.a) && b.forall(_ == obj.b)
}
} else Nil
}
Everything looks fine to me except the filter. Consider:
objs filter { obj =>
( a.isEmpty || a.get == obj.a ) && ( b.isEmpty || b.get == obj.b )
}
In cases like these where you want the behavior in the "None" case to "mix in" with the behavior for the "Some" case, map is kind of useless.
You do not need to test whether each parameter is a Some over and over for each object in the list -- you only need to do that once. map each argument to a function that performs the test, and then flatten those into a list of filter functions. Then if there were filters, reduce them into a single filter and apply it to the list:
def judge(optA: Option[Int], optB: Option[Int]) = {
val foos = List(Foo(1,2), Foo(3,4))
val filters = List(
optA map (a => (foo: Foo) => foo.a == a),
optB map (b => (foo: Foo) => foo.b == b)
).flatten
if (filters.isEmpty) foos else foos filter filters.reduce{
(f1, f2) => (foo: Foo) => f1(foo) && f2(foo)
}
}
Using a Vector to hold the objects would improve performance, both by dramatically reducing the number of objects allocated and by avoiding the implicit reverse that would be done after filtering a List.
Much like this question:
Functional code for looping with early exit
Say the code is
def findFirst[T](objects: List[T]):T = {
for (obj <- objects) {
if (expensiveFunc(obj) != null) return /*???*/ Some(obj)
}
None
}
How to yield a single element from a for loop like this in scala?
I do not want to use find, as proposed in the original question, i am curious about if and how it could be implemented using the for loop.
* UPDATE *
First, thanks for all the comments, but i guess i was not clear in the question. I am shooting for something like this:
val seven = for {
x <- 1 to 10
if x == 7
} return x
And that does not compile. The two errors are:
- return outside method definition
- method main has return statement; needs result type
I know find() would be better in this case, i am just learning and exploring the language. And in a more complex case with several iterators, i think finding with for can actually be usefull.
Thanks commenters, i'll start a bounty to make up for the bad posing of the question :)
If you want to use a for loop, which uses a nicer syntax than chained invocations of .find, .filter, etc., there is a neat trick. Instead of iterating over strict collections like list, iterate over lazy ones like iterators or streams. If you're starting with a strict collection, make it lazy with, e.g. .toIterator.
Let's see an example.
First let's define a "noisy" int, that will show us when it is invoked
def noisyInt(i : Int) = () => { println("Getting %d!".format(i)); i }
Now let's fill a list with some of these:
val l = List(1, 2, 3, 4).map(noisyInt)
We want to look for the first element which is even.
val r1 = for(e <- l; val v = e() ; if v % 2 == 0) yield v
The above line results in:
Getting 1!
Getting 2!
Getting 3!
Getting 4!
r1: List[Int] = List(2, 4)
...meaning that all elements were accessed. That makes sense, given that the resulting list contains all even numbers. Let's iterate over an iterator this time:
val r2 = (for(e <- l.toIterator; val v = e() ; if v % 2 == 0) yield v)
This results in:
Getting 1!
Getting 2!
r2: Iterator[Int] = non-empty iterator
Notice that the loop was executed only up to the point were it could figure out whether the result was an empty or non-empty iterator.
To get the first result, you can now simply call r2.next.
If you want a result of an Option type, use:
if(r2.hasNext) Some(r2.next) else None
Edit Your second example in this encoding is just:
val seven = (for {
x <- (1 to 10).toIterator
if x == 7
} yield x).next
...of course, you should be sure that there is always at least a solution if you're going to use .next. Alternatively, use headOption, defined for all Traversables, to get an Option[Int].
You can turn your list into a stream, so that any filters that the for-loop contains are only evaluated on-demand. However, yielding from the stream will always return a stream, and what you want is I suppose an option, so, as a final step you can check whether the resulting stream has at least one element, and return its head as a option. The headOption function does exactly that.
def findFirst[T](objects: List[T], expensiveFunc: T => Boolean): Option[T] =
(for (obj <- objects.toStream if expensiveFunc(obj)) yield obj).headOption
Why not do exactly what you sketched above, that is, return from the loop early? If you are interested in what Scala actually does under the hood, run your code with -print. Scala desugares the loop into a foreach and then uses an exception to leave the foreach prematurely.
So what you are trying to do is to break out a loop after your condition is satisfied. Answer here might be what you are looking for. How do I break out of a loop in Scala?.
Overall, for comprehension in Scala is translated into map, flatmap and filter operations. So it will not be possible to break out of these functions unless you throw an exception.
If you are wondering, this is how find is implemented in LineerSeqOptimized.scala; which List inherits
override /*IterableLike*/
def find(p: A => Boolean): Option[A] = {
var these = this
while (!these.isEmpty) {
if (p(these.head)) return Some(these.head)
these = these.tail
}
None
}
This is a horrible hack. But it would get you the result you wished for.
Idiomatically you'd use a Stream or View and just compute the parts you need.
def findFirst[T](objects: List[T]): T = {
def expensiveFunc(o : T) = // unclear what should be returned here
case class MissusedException(val data: T) extends Exception
try {
(for (obj <- objects) {
if (expensiveFunc(obj) != null) throw new MissusedException(obj)
})
objects.head // T must be returned from loop, dummy
} catch {
case MissusedException(obj) => obj
}
}
Why not something like
object Main {
def main(args: Array[String]): Unit = {
val seven = (for (
x <- 1 to 10
if x == 7
) yield x).headOption
}
}
Variable seven will be an Option holding Some(value) if value satisfies condition
I hope to help you.
I think ... no 'return' impl.
object TakeWhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else
seq(seq.takeWhile(_ == null).size)
}
object OptionLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T], index: Int = 0): T = if (seq.isEmpty) null.asInstanceOf[T] else
Option(seq(index)) getOrElse func(seq, index + 1)
}
object WhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else {
var i = 0
def obj = seq(i)
while (obj == null)
i += 1
obj
}
}
objects iterator filter { obj => (expensiveFunc(obj) != null } next
The trick is to get some lazy evaluated view on the colelction, either an iterator or a Stream, or objects.view. The filter will only execute as far as needed.