This might be a naive question and I am sorry for that. I am studying Scala Futures and stumbled on below code:
object Main extends App {
def await[T](f: Future[T]) = Await.result(f, 10.seconds)
def f(n: Int): Future[Int] = Future {n + 1}
def g(n: Int): Future[Int] = Future {n * 2}
def h(n: Int): Future[Int] = Future {n - 1}
def doAllInOrder[T](f: (T => Future[T])*): T => Future[T] = {
f.reduceLeft((a,b) => x => a(x).flatMap(y => b(y)))
}
println(await(doAllInOrder(f, g, h)(10))) // 21
}
I know how reduceLeft works when it is applied on Collections. But in the above example, as I understand, in the first pass of reduceLeft value x i.e. 10 is applied to Function a and the result is applied to Function b using flatMap which eventually return Future[Int] (say, I call it result). In the next pass of reduceLeft the result and Function h has to be used, but here is I am troubled.
The result is actually an already executed Future, but the reduceLeft next pass expects a Function which returns a Future[Int].Then how it is working?
Another thing I am not able to understand how each pass sending its result to next pass of reduceLeft, i.e. how x is getting it's value in subsequent passes.
Though both of my confusions are interrelated and a good explanation may help clear my doubt.
Thanks in advance.
You have to think about reduceLeft to be independent from Future execution. reduceLeft creates a new function by combining two given ones and that's it.
reduceLeft is applied to a Seq of T => Future[T]. So, it's just simple iteration from left to right over sequence of functions taking first and second elements and reducing it to one single value, reducing this value with the 3rd element and so on. Eventually, having just two element left that are reduced to a single one.
The result of reduceLeft has to be of the same type as the type of elements in the collection. In your case, it's function T => Future[T].
Let's understand what (a,b) => x => a(x).flatMap(y => b(y)) is doing
This means the following. Given functions a and b, create a function that combines function a with b. Mathematically it's c(x)=b(a(x)).
Now, a and b are functions returning futures. And futures can be chained with the help of map/flatMap methods.
You should read x => a(x).flatMap(y => b(y)) as
Given an input x, apply function a(x), this results in a Future, when this future is completed, take result y and apply function b(y), this results in a new Future. This is the result of function c.
Note: value x is Int at all times. It is the input parameter for your new reduced function.
If it's still not clear, let's address the points of confusions
The result is actually an already executed Future.
No future is guaranteed to be executed at any point here. map and flatMap are non blocking operation and it applies functions to a Future. The result of this application is still a Future.
Another thing I am not able to understand how each pass sending its
result to next pass of reduceLeft, i.e. how x is getting it's value in
subsequent passes.
This is easier to understand when just having integers in a collection. Given following code
Seq(1, 2, 5, 10).reduceLeft(_ - _)
It will take 1, 2 and apply - function, this will result in -1. It will then combine -1 and 5 resulting in -6. And finally, -6 with 10 resulting in -16.
Related
def flatMap[A,B](f: Rand[A])(g: A => Rand[B]): Rand[B] =rng => {
val (a, r1) = f(rng)
g(a)(r1)
}
I am confused by g(a)(r1) because g is supposed to take only one argument, so why r1?
I don't exactly know the Rand[A] structure, so this is partially guessing. What you are returning in this function is a Rand[B] and when you implement the function you start of with defining an argument rng of an anonymous function. This tells me that Rand is probably some kind of function itself.
In the second line (val (a, r1) = f(rng)) you apply the value rng to the instance f: Rand[A]. In the resulting tuple, a has type A.
Note with this that f(rng) is equal to explicitly calling apply: f.apply(rng).
You can use the value a to get a Rand[B] by applying this value to the function g. So, val rb: Rand[B] = g(a) (or g.apply(a)).
Now, you don't want to return an instance of Rand[B] in this anonymous function! Instead you need to apply the previous result r1 to this instance rb. So, you get rb(r1) or rb.apply(r1). Substituting rb with g(a) or g.apply(a) gives you g(a)(r1) or g.apply(a).apply(r1).
To summarize, as you said, g is supposed to only take one argument, which results in an instance of Rand[B], but that is not the expected return type here. You need to apply the result of the previous computation to this new computation to get the expected result.
An infinite stream:
val ones: Stream[Int] = Stream.cons(1, ones)
How is it possible for a value to be used in its own declaration? It seems this should produce a compiler error, yet it works.
It's not always a recursive definition. This actually works and produces 1:
val a : Int = a + 1
println(a)
variable a is created when you type val a: Int, so you can use it in the definition. Int is initialized to 0 by default. A class will be null.
As #Chris pointed out, Stream accepts => Stream[A] so a bit another rules are applied, but I wanted to explain general case. The idea is still the same, but the variable is passed by-name, so this makes the computation recursive. Given that it is passed by name, it is executed lazily. Stream computes each element one-by-one, so it calls ones each time it needs next element, resulting in the same element being produces once again. This works:
val ones: Stream[Int] = Stream.cons(1, ones)
println((ones take 10).toList) // List(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)
Though you can make infinite stream easier: Stream.continually(1) Update As #SethTisue pointed out in the comments Stream.continually and Stream.cons are two completely different approaches, with very different results, because cons takes A when continually takes =>A, which means that continually recomputes each time the element and stores it in the memory, when cons can avoid storing it n times unless you convert it to the other structure like List. You should use continually only if you need to generate different values. See #SethTisue comment for details and examples.
But notice that you are required to specify the type, the same as with recursive functions.
And you can make the first example recursive:
lazy val b: Int = b + 1
println(b)
This will stackoverflow.
Look at the signature of Stream.cons.apply:
apply[A](hd: A, tl: ⇒ Stream[A]): Cons[A]
The ⇒ on the second parameter indicates that it has call-by-name semantics. Therefore your expression Stream.cons(1, ones) is not strictly evaluated; the argument ones does not need to be computed prior to being passed as an argument for tl.
The reason this does not produce a compiler error is because both Stream.cons and Cons are non-strict and lazily evaluate their second parameter.
ones can be used in it's own definition because the object cons has an apply method defined like this:
/** A stream consisting of a given first element and remaining elements
* #param hd The first element of the result stream
* #param tl The remaining elements of the result stream
*/
def apply[A](hd: A, tl: => Stream[A]) = new Cons(hd, tl)
And Cons is defined like this:
final class Cons[+A](hd: A, tl: => Stream[A]) extends Stream[A]
Notice that it's second parameter tl is passed by name (=> Stream[A]) rather than by value. In other words, the parameter tl is not evaluated until it is used in the function.
One advantage to using this technique is that you can compose complex expressions that may be only partially evaluated.
I just wanted to clarify something about partially defined functions in Scala. I looked at the docs and it said the type of a partial function is PartialFunction[A,B], and I can define a partial function such as
val f: PartialFunction[Any, Int] = {...}
I was wondering, for the types A and B, is A a parameter, and B a return type? If I have multiple accepted types, do I use orElse to chain partial functions together?
In the set theoretic view of a function, if a function can map every value in the domain to a value in the range, we say that this function is a total function. There can be situations where a function cannot map some element(s) in the domain to the range; such functions are called partial functions.
Taking the example from the Scala docs for partial functions:
val isEven: PartialFunction[Int, String] = {
case x if x % 2 == 0 => x+" is even"
}
Here a partial function is defined since it is defined to only map an even integer to a string. So the input to the partial function is an integer and the output is a string.
val isOdd: PartialFunction[Int, String] = {
case x if x % 2 == 1 => x+" is odd"
}
isOdd is another partial function similarly defined as isEven but for odd numbers. Again, the input to the partial function is an integer and the output is a string.
If you have a list of numbers such as:
List(1,2,3,4,5)
and apply the isEven partial function on this list you will get as output
List(2 is even, 4 is even)
Notice that not all the elements in the original list have been mapped by the partial function. However, there may be situations where you want to apply another function in those cases where a partial function cannot map an element from the domain to the range. In this case we use orElse:
val numbers = sample map (isEven orElse isOdd)
And now you will get as output:
List(1 is odd, 2 is even, 3 is odd, 4 is even, 5 is odd)
If you are looking to set up a partial function that, in effect, takes multiple parameters, define the partial function over a tuple of the parameters you'll be feeding into it, eg:
val multiArgPartial: PartialFunction[(String, Long, Foo), Int] = {
case ("OK", _, Foo("bar", _)) => 0 // Use underscore to accept any value for a given parameter
}
and, of course, make sure you pass arguments to it as tuples.
In addition to other answers, if by "multiple accepted types" you mean that you want the same function accept e.g. String, Int and Boolean (and no other types), this is called "union types" and isn't supported in Scala currently (but is planned for the future, based on Dotty). The alternatives are:
Use the least common supertype (Any for the above case). This is what orElse chains will do.
Use a type like Either[String, Either[Int, Boolean]]. This is fine if you have two types, but becomes ugly quickly.
Encode union types as negation of intersection types.
Disclaimer: the code snippet below relates to one of ongoing Coursera courses.
Let's consider it's posted just for a learning purpose and should not be used for submitting as a solution for one's homework assignment.
As the comment below states, we need to transform a list of Futures to a single Future of a list. More than that, the resulting Future should fail if at least one of input futures failed.
I met the following implementation and I don't understand it completely.
/** Given a list of futures `fs`, returns the future holding the list of values of all the futures from `fs`.
* The returned future is completed only once all of the futures in `fs` have been completed.
* The values in the list are in the same order as corresponding futures `fs`.
* If any of the futures `fs` fails, the resulting future also fails.
*/
def all[T](fs: List[Future[T]]): Future[List[T]] =
fs.foldRight(Future(Nil:List[T]))((f, fs2) =>
for {
x <- f
xs <- fs2
} yield (x::xs))
In particular, I don't understand the next things in it:
Where does Future[T] -> T transformation happen? It looks like xs <- fs2 is the only place we touch initial Futures, and each of xs type should be Future[T] (but somehow it becomes just T).
How are failures handled? It looks like the resulting Future object does fail when one of the input Futures fails.
1) Say f is a Future[T], then writing
for {
t <- f
} yield List(t)
will store the result of the Future f in t - therefor t is of type T. The yield turns it into a List[T], and the type of the whole for-comprehension ends up being Future[List[T]]. So the for-comprehension is where you extract your Ts from your Futures, do something with them, and put them back in a Future (OK, I'm simplifying a little bit here).
It's equivalent to
f.map(t => List(t))
2) If your Future f contains a Failure, then the for-comprehension will just return this failed Future instead of executing the yield.
As a general note, for-comprehension in Scala is just sugar that can be rewritten with map, flatMap, filter, foreach.
I'm an English-speaking right-hander, so normally I foldLeft, but each step of the fold looks like:
Fn flatMap ((x: T) => Fs map (xs => x :: xs))
Your value is x.
The function is applied on success, which explains why a failure stops you cold:
scala> timed(Await.ready(all(List(Future{Thread sleep 5*1000; 1},Future(2),Future{Thread sleep 10*1000; 3})), Duration.Inf))
res0: (Long, scala.concurrent.Awaitable[List[Int]]) = (10002419021,scala.concurrent.impl.Promise$DefaultPromise#2a8025a0)
scala> timed(Await.ready(all(List(Future{Thread sleep 5*1000; 1},Future(???),Future{Thread sleep 10*1000; 3})), Duration.Inf))
res1: (Long, scala.concurrent.Awaitable[List[Int]]) = (5000880298,scala.concurrent.impl.Promise$DefaultPromise#3750d517)
Notice that the failing version short-circuits.
See the ScalaDoc for flatMap for both bits of information.
Edit: I was speaking cautiously because it is Coursera work, but more plainly, this requirement is not met: "The returned future is completed only once all of the futures in fs have been completed."
In my Scala course an example has given. It was about finding a more generalized function, which can be used to define an arithmetic summation function and an arithmetic production function. Here are the functions that should be generalized.
def sum(f:Int=>Int)(a:Int,b:Int):Int ={
if(a>b) 0
else f(a) + sum(f)(a+1,b)
}
def product(f:Int=>Int)(a:Int,b:Int):Int={
if(a>b)1
else f(a)*product(f)(a+1,b)
}
To generalize these functions the teacher gave such a function :
def mapReduce(f:Int=>Int,combine: (Int,Int)=>Int, zero:Int)(a:Int,b:Int):Int ={
if(a>b) zero
else combine(f(a),mapReduce(f, combine, zero)(a+1, b))
}
So mapReduce function can be used to generalize sum and product functions as follows:
def sumGN(f:Int=>Int)(a:Int,b:Int) = mapReduce(f, (x,y)=>(x+y), 0)(a, b)
def productGN(f:Int=>Int)(a:Int,b:Int) = mapReduce(f, (x,y)=>(x*y), 1)(a, b)
I took a look at the definition of map reduce in functional programming but I have a hard time why the generalized function has been named as map reduce above. I can not grasp the relation. Any help will make my very happy.
Regards
Functional programming usually has three central operators: map, reduce (sometimes called fold), and filter.
Map takes a list and an operation and produces a list containing the operation applied to everything in the first list.
Filter takes a list and a test and produces another list containing only the elements that pass the test.
Reduce (or fold) takes a list, an operation, and an initial value and applies the operation to the initial value and the elements in the list, passing the output into itself along with the next list item, producing the operational sum of the list.
If, for example, your list is [2,3,4,5,6,7], your initial value is 1, and your operation is addition, reduction will behave in the following way:
Reduce([2,3,4,5,6,7], +, 1) = ((((((initial + 2) + 3) + 4) + 5) + 6) + 7)
Your instructor may be calling it mapReduce because this is the paradigm's name, though simply reduce would be sufficient as well.
If you're curious as to the significance of his name, you should ask him. He is your instructor and all.
This is by no means an exact explanation (names are fuzzy anyway) but here’s an alternative definition:
def mapReduce(f: Int => Int, combine: (Int, Int) => Int, zero: Int)(a: Int, b: Int): Int ={
if (a > b) zero
else (a to b).map(f).reduce(combine)
}
Do you see the link?
mapReduce's mapping function is f in the question, though there's never an example of its definition. For sum and product it would be the identity function, but if you were summing the squares then the mapping function would be the square function.
mapReduce's reducer function is combine, wherein we are reducing a tuple of accumulator+value to a new accumulator for the next recursion.
I think the missing link besides the code not being very clear is to treat numbers as collections (e.g., 3 is a collection of three 1s). This is quite unusual and I don't know what it buys you but it could be that your teacher will use the analogy between numbers and collections for something more profound later.
Is this from Odersky's coursera course?