//this class holds method run that let us give a state and obtain from it an
//output value and next state
case class State[S, +A](run: S => (A, S)) {
//returns a state for which the run function will return (b,s) instead of (a,s)
def map[B](f: A => B): State[S, B] =
flatMap(a => unit(f(a)))
//returns a state for which the run function will return (f(a,b),s)
def map2[B, C](sb: State[S, B])(f: (A, B) => C): State[S, C] =
flatMap(a => sb.map(b => f(a, b)))
def flatMap[B](f: A => State[S, B]): State[S, B] =
State(s => {
val (a, s1) = run(s)
val sB = f(a)
sB.run(s1)
})
}
I am new to functional programming and I am dizzied by this code, I tried to understand what does flatMap really does but it looks very abstract to me that I couldn't understand it. For example here we defined map in terms of flatMap, but I can't understand how can we explain the implementation I mean how does it work?
This chapter is about pure functional state, and describes how the state is being updated in a sequence of operations in a functional way, that is to say, by means of functional composition. So you don´t have to create a global state variable that is mutated by a sequence of method executions, as you probably would do in a OOP fashion.
Having said that, this wonderful book explains each concept declaring an abstraction to represent an effect, and some primitives which are going to be used to build more complex functions. In this chapter, the effect is the State. It is represented as a function that takes a previous state and yields a value and a new state, as you can see with the run function declaration. So if you want to compose functions that propagate some state in a pure functional fashion, all of them need to return a new functional State.
With flatMap, which is a primitive of the State type, you can compose functions and, at the same time, propagate some internal state.
For example, if you have two functions:
case class Intermediate(v: Double = 0)
def execute1(a: Int): State[Int, Intermediate] = {
State { s =>
/** Do whatever */
(Intermediate(a * a), s + 1)
}
}
def execute2(a: Int): State[Int, Intermediate] = {
State { s =>
/** Do whatever */
(Intermediate(a * a), s + 1)
}
}
As you can see each function consist in a State function that takes a previous state, executes some logic, and returns a tuple of Int, which is going to be used to track the number of functions that have been executed, and a Intermediate value that we use to yield the execution of every function.
Now, if you apply flat map to execute both functions sequentially:
val result: State[Int, Intermediate] =
execute1(2) flatMap (r => execute2(r.v.toInt))
the result val is another State that you can see as the point of entry to run the pipeline:
/** 0 would be the initial state **/
result.run(0)
the result is:
(Intermediate(16.0),2)
How this works under the hood:
First you have the first function execute1. This function returns a State[Intermediate, Int]. I will use Intermediate to store the result of each execution, that would be the yielded value, and an Int to count the number of functions that are going to be executed in the flatMap sequential composition.
Then I apply flatMap after the function call, which is the flatMap of the State type.
Remember the definition:
def flatMap[B](f: A => State[S, B]): State[S, B] =
State(s => {
/* This function is executed in a State context */
val (a, s1) = run(s)
val sB = f(a)
sB.run(s1)
})
The execution is inside a state context(note that the flatMap function is inside the State type) which is be the first state function of the execute1 function. Maybe this representation is easier:
val st1 = execute1(2)
st1.flatMap(r => execute2(r.v.toInt))
So when run(s) is executed, it actually executes the function state returned by the execute1(2) function. It gives the first (a, s1) result.
Then you have to execute the second function, execute2. You can visualize this function as the input parameter of the flatMap function. This function is executed with the yielded value of the execute1(2) function (the Intermediate type, left side of the tuple). This execution returns another state function that uses the state(right value of the tuple) of the execution1, to create the final tuple with the resulting yielded value of the execute2 function and the int corresponding to the number of executions. Note, that in each function we increment the internal state in one unit.
Note that the result is another State. This has a run method that waits for the initial state.
I have tried to explain the workflow of function calls... it is a little bit hard if you are not used to think in a functional style, but it is only functional composition, it becomes familiar after a while.
Related
Is there a map like function in Scala that renders: M[A] -> M[B] but also allows you to specify some context info kinda like in scanLeft? For instance, something like:
def mapWithContext(context: C, f: (C, A) => (C, B)): M[B]
f would be passed 1) the context info of type C 2) the current element in the list of type A
and it would then returns 1) the updated context info C 2) the transformed list element of type B. However, unlike scanLeft it would then discard the context info from the collection it is building and only return the B's
Not exactly, but the new (Scala 2.13) unfold() method comes close. It's available on almost all collection types, mutable and immutable. Here's the profile for List.
def unfold(init: S)(f: (S) => Option[(A, S)]): List[A]
It takes an initial state and continually applies f() until it returns None.
So what you could to is make the initial state a tuple (S, M[A]) and make the f() function f:(S,M[A]) => Option[(B, (S,M[A]))]. The f() code would peal off the head of M[A] to produce each B element until M[A] is empty, when f() returns None.
I have been trying to understand the State Monad. Not so much how it is used, though that is not always easy to find, either. But every discussion I find of the State Monad has basically the same information and there is always something I don't understand.
Take this post, for example. In it the author has the following:
case class State[S, A](run: S => (A, S)) {
...
def flatMap[B](f: A => State[S, B]): State[S, B] =
State(s => {
val (a, t) = run(s)
f(a) run t
})
...
}
I can see that the types line up correctly. However, I don't understand the second run at all.
Perhaps I am looking at the whole purpose of this monad incorrectly. I got the impression from the HaskellWiki that the State monad was kind of like a state-machine with the run allowing for transitions (though, in this case, the state-machine doesn't really have fixed state transitions like most state machines). If that is the case then in the above code (a, t) would represent a single transition. The application of f would represent a modification of that value and State (generating a new State object). That leaves me completely confused as to what the second run is all about. It would appear to be a second 'transition'. But that doesn't make any sense to me.
I can see that calling run on the resulting State object produces a new (A, S) pair which, of course, is required for the types to line up. But I don't really see what this is supposed to be doing.
So, what is really going on here? What is the concept being modeled here?
Edit: 12/22/2015
So, it appears I am not expressing my issue very well. Let me try this.
In the same blog post we see the following code for map:
def map[B](f: A => B): State[S, B] =
State(s => {
val (a, t) = run(s)
(f(a), t)
})
Obviously there is only a single call to run here.
The model I have been trying to reconcile is that a call to run moves the state we are keeping forward by a single state-change. This seems to be the case in map. However, in flatMap we have two calls to run. If my model was correct that would result in 'skipping over' a state change.
To make use of the example #Filppo provided below, the first call to run would result in returning (1, List(2,3,4,5)) and the second would result in (2, List(3,4,5)), effectively skipping over the first one. Since, in his example, this was followed immediately by a call to map, this would have resulted in (Map(a->2, b->3), List(4,5)).
Apparently that is not what is happening. So my whole model is incorrect. What is the correct way to reason about this?
2nd Edit: 12/22/2015
I just tried doing what I said in the REPL. And my instincts were correct which leaves me even more confused.
scala> val v = State(head[Int]).flatMap { a => State(head[Int]) }
v: State[List[Int],Int] = State(<function1>
scala> v.run(List(1,2,3,4,5))
res2: (Int, List[Int]) = (2,List(3, 4, 5))
So, this implementation of flatMap does skip over a state. Yet when I run #Filippo's example I get the same answer he does. What is really happening here?
To understand the "second run" let's analyse it "backwards".
The signature def flatMap[B](f: A => State[S, B]): State[S, B] suggests that we need to run a function f and return its result.
To execute function f we need to give it an A. Where do we get one?
Well, we have run that can give us A out of S, so we need an S.
Because of that we do: s => val (a, t) = run(s) ....
We read it as "given an S execute the run function which produces us A and a new S. And this is our "first" run.
Now we have an A and we can execute f. That's what we wanted and f(a) gives us a new State[S, B].
If we do that then we have a function which takes S and returns Stats[S, B]:
(s: S) =>
val (a, t) = run(s)
f(a) //State[S, B]
But function S => State[S, B] isn't what we want to return! We want to return just State[S, B].
How do we do that? We can wrap this function into State:
State(s => ... f(a))
But it doesn't work because State takes S => (B, S), not S => State[B, S].
So we need to get (B, S) out of State[B, S].
We do it by just calling its run method and providing it with the state we just produced on the previous step!
And it is our "second" run.
So as a result we have the following transformation performed by a flatMap:
s => // when a state is provided
val (a, t) = run(s) // produce an `A` and a new state value
val resState = f(a) // produce a new `State[S, B]`
resState.run(t) // return `(S, B)`
This gives us S => (S, B) and we just wrap it with the State constructor.
Another way of looking at these "two runs" is:
first - we transform the state ourselves with "our" run function
second - we pass that transformed state to the function f and let it do its own transformation.
So we kind of "chaining" state transformations one after another. And that's exactly what monads do: they provide us with the ability to schedule computation sequentially.
The state monad boils down to this function from one state to another state (plus A):
type StatefulComputation[S, +A] = S => (A, S)
The implementation mentioned by Tony in that blog post "capture" that function into run of the case class:
case class State[S, A](run: S => (A, S))
The flatmap implementation to bind a state to another state is calling 2 different runs:
// the `run` on the actual `state`
val (a: A, nextState: S) = run(s)
// the `run` on the bound `state`
f(a).run(nextState)
EDIT Example of flatmap between 2 State
Considering a function that simply call .head to a List to get A, and .tail for the next state S
// stateful computation: `S => (A, S)` where `S` is `List[A]`
def head[A](xs: List[A]): (A, List[A]) = (xs.head, xs.tail)
A simple binding of 2 State(head[Int]):
// flatmap example
val result = for {
a <- State(head[Int])
b <- State(head[Int])
} yield Map('a' -> a,
'b' -> b)
The expect behaviour of the for-comprehension is to "extract" the first element of a list into a and the second one in b. The resulting state S would be the remaining tail of the run list:
scala> result.run(List(1, 2, 3, 4, 5))
(Map(a -> 1, b -> 2),List(3, 4, 5))
How? Calling the "stateful computation" head[Int] that is in run on some state s:
s => run(s)
That gives the head (A) and the tail (B) of the list. Now we need to pass the tail to the next State(head[Int])
f(a).run(t)
Where f is in the flatmap signature:
def flatMap[B](f: A => State[S, B]): State[S, B]
Maybe to better understand what is f in this example, we should de-sugar the for-comprehension to:
val result = State(head[Int]).flatMap {
a => State(head[Int]).map {
b => Map('a' -> a, 'b' -> b)
}
}
With f(a) we pass a into the function and with run(t) we pass the modified state.
I have accepted #AlexyRaga's answer to my question. I think #Filippo's answer was very good as well and, in fact, gave me some additional food for thought. Thanks to both of you.
I think the conceptual difficulty I was having was really mostly to do with 'what does the run method 'mean'. That is, what is its purpose and result. I was looking at it as a 'transition' function (from one state to the next). And, after a fashion, that is what it does. However, it doesn't transition from a given (this) state to the next state. Instead, it takes an initial State and returns the (this) state's value and a new 'current' state (not the next state in the state-transition sequence).
That is why the flatMap method is implemented the way it is. When you generate a new State then you need the current value/state pair from it based on the passed-in initial state which can then be wrapped in a new State object as a function. You are not really transitioning to a new state. Just re-wrapping the generated state in a new State object.
I was too steeped in traditional state machines to see what was going on here.
Thank, again, everyone.
I'm working with this code:
case class State[S, +A](run: S => (A, S)) {
...
def flatMap[B](f: A => State[S, B]): State[S, B] = State(s => {
val (a, s1) = run(s)
f(a).run(s1)
})
...
}
This is an abstraction for working with purely functional state, from §6 of FP in Scala. run is function parameter that takes a state and emits a tuple of a value and the new state.
My question is around the syntax s => in this section:
... B] = State(s => { ...
This appears to be using the State 'constructor' (ie apply) to build a new State object. But what does the s represent? Is it an 'anonymous' State, representing any State instance? If so, how is it different from this? Or does s correspond to the input parameter of run ie the S from:
... (run: S => ....
And why would I be using a constructor for a function definition? Notice that the last symbol of the flatMap definition is a ) not a }, that is closing the State apply constructor.
This scenario is a bit different than the standard
case class Person(name: String)
scenario, so I thought I'd ask...
Your second assumption is correct, s corresponds to the input parameter of run function, ie the S, which represents an actual state passing through the chain. So s => {...} is just a lambda definition of type S => (A, S). Lambdas and functions in Scala are first-class citizens (values), so you can pass them as parameters to hold inside some another type (including some monad).
Here the function, which produces new raw state S (and new result A), is wrapped (see return operation) to the monad State, which is implemented as case class. We need it to define flatMap operation (see bind) over the monad.
To make it more clear about the passing function as parameter, the code may be rewritten to:
case class State[S, +A](run: S => (A, S)) {
def flatMap[B](f: A => State[S, B]): State[S, B] = {
def newState(s: S) = {
val (a, s1) = run(s)
f(a).run(s1)
}
State(newState _)
}
}
So, according to the monad definition:
State[S, +A] is a type constructor which takes two plain types (S and covariant A) and returns monadic type State
State.apply(run: S => (A, S)) function takes a plain function and returns (lifts into) monadic container State, so it is a "return" operator of the monad
State.flatMap[B](f: A => State[S, B]): State[S, B] is corresponding to the "bind" operator
Case class is used just to have explicit "return" function (apply) instead of using new operator.
(It seems like I ask too many basic questions, but Scala is very difficult, especially when it comes to the language feature or syntactic sugar, and I come from a Java background..)
I am watching this video: Scalaz State Monad on YouTube. The lecturer wrote these code:
trait State[S, +A] {
def run(initial: S):(S, A)
def map[B](f: A=>B): State[S, B] =
State { s =>
val (s1, a) = run(s)
(s1, f(a))
}
He said the State on the forth line is a "lambda" expression (I may have misheard). I'm a bit confused about this. He did have an object State, which looks like this:
Object State {
def apply[S, A](f: S=>(S, A)): State[S, A] =
new State[S, A] {
def run(i: S) = f(i)
}
}
The lecturer said he was invoking the apply method of Object State by invoking like that, and yes he did pass in a function that returns a correct tuple, but what is that { s=>? Where the hell did he get the s and is this about the implicit mechanism in Scala?
Also the function that is passed in map function changes A to B already, how can it still pass the generics check (from [S, A] to [S, B] assuming A and B are two different types, or are they the same type?If true, why use B not A in map function?)?
The video link is here and is set to be at 18:17
http://youtu.be/Jg3Uv_YWJqI?t=18m17s
Thank you (Monad is a bit daunting to anyone that's not so familiar with functional programming)
You are on right way here.
He passes the function as parameter, but what is the function in general? Function takes argument and return a value. Returned value here is tuple (s1, f(a)), but also we need to reffer to function argument. s in your example is function argument.
It is same as
def map[B](f: A=>B): State[S, B] = {
def buildState(s: S) = {
val (s1, a) = run(s)
(s1, f(a))
}
State(buildState)
}
But, since we dont need this function anywhere else we just inline it in apply call.
I would like to try partial functions with a deep pattern matching use case. This initially (of course) didn't work after applying Some(Some(3)), but seemed defined instead :
def deepTest : PartialFunction [Option[Option[Int]], Int] = {
case Some(v) => v match {
case None => 3
}
case None => 1
}
and I thought that by decoupling the nested pattern matching, things would be easier:
def deepTestLvl1 : PartialFunction [Option[Option[Int]], Option[Int]] = {
case Some(v) => v
case None => Some(1)
}
def deepTestLvl2 : PartialFunction [Option[Int], Int] = {
case None => 3
}
but the result was the following:
scala> (deepTestLvl1 andThen deepTestLvl2) isDefinedAt(Some(Some(3)))
res24: Boolean = true
and after applying:
scala> (deepTestLvl1 andThen deepTestLvl2) (Some(Some(3)))
scala.MatchError: Some(3) (of class scala.Some)
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:248)
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:246)
at $anonfun$deepTestLvl2$1.applyOrElse(<console>:7)
at $anonfun$deepTestLvl2$1.applyOrElse(<console>:7)
....
at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:83)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:96)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:105)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
Am I doing something incorrectly? Shouldn't the isDefinedAt be called twice when I composed sequentially deepTestLvl{1,2} and give me the correct answer?
Very good question.
Let's check the source and see what's happening under the covers:
override def andThen[C](k: B => C): PartialFunction[A, C] =
new AndThen[A, B, C] (this, k)
We can observe here that andThen doesn't even expect a Partial Function, any Function that transforms the result will do. Your code works, because: trait PartialFunction[-A, +B] extends (A => B). This can actually be found in the documentation:
def andThen[C](k: (B) ⇒ C): PartialFunction[A, C]
Composes this partial function with a transformation function that gets applied to results of this partial function.
C the result type of the transformation function.
k the transformation function
returns a partial function with the same domain as this partial function, which maps arguments x to k(this(x)).
So there's currently no way to chain PartialFunctions in the way you would like, because as Robin said, it would require applying the function. Next to being computationally expensive it could also have side effects, which is a bigger problem.
Update
Whipped together an implementation you're looking for. Use it cautiously! As I already mentioned, if your code has side effects it will cause problems:
implicit class PartialFunctionExtension[-A, B](pf: PartialFunction[A, B]) {
def andThenPf[C](pf2: PartialFunction[B, C]) = new PfAndThen(pf, pf2)
class PfAndThen[+C](pf: PartialFunction[A, B], nextPf: PartialFunction[B, C]) extends PartialFunction[A, C] {
def isDefinedAt(x: A) = pf.isDefinedAt(x) && nextPf.isDefinedAt(pf.apply(x))
def apply(x: A): C = nextPf(pf(x))
}
}
Trying it out:
deepTestLvl1.andThenPf(deepTestLvl2).isDefinedAt(Some(Some(3))) // false
deepTestLvl1.andThenPf(deepTestLvl2).isDefinedAt(Some(None)) // true
deepTestLvl1.andThenPf(deepTestLvl2).apply(Some(None)) // 3
The reason why isDefinedAt on a PartialFunction produced by andThen returns inconsistent results is that it doesn't actually apply the first partial function to its argument, which could be an expensive operation.
This behaviour is likely to trip people up and isn't documented - you might want to submit a patch to add documentation for that.
P.S. My guess is that the reason for deepTest to behave as it does, is that the outermost match in the source code of a partial function definition, and only the outermost match, is considered for definedness purposes - but you'd have to check the source code of scalac to be sure, I think.