Is there a map like function in Scala that renders: M[A] -> M[B] but also allows you to specify some context info kinda like in scanLeft? For instance, something like:
def mapWithContext(context: C, f: (C, A) => (C, B)): M[B]
f would be passed 1) the context info of type C 2) the current element in the list of type A
and it would then returns 1) the updated context info C 2) the transformed list element of type B. However, unlike scanLeft it would then discard the context info from the collection it is building and only return the B's
Not exactly, but the new (Scala 2.13) unfold() method comes close. It's available on almost all collection types, mutable and immutable. Here's the profile for List.
def unfold(init: S)(f: (S) => Option[(A, S)]): List[A]
It takes an initial state and continually applies f() until it returns None.
So what you could to is make the initial state a tuple (S, M[A]) and make the f() function f:(S,M[A]) => Option[(B, (S,M[A]))]. The f() code would peal off the head of M[A] to produce each B element until M[A] is empty, when f() returns None.
Related
//this class holds method run that let us give a state and obtain from it an
//output value and next state
case class State[S, +A](run: S => (A, S)) {
//returns a state for which the run function will return (b,s) instead of (a,s)
def map[B](f: A => B): State[S, B] =
flatMap(a => unit(f(a)))
//returns a state for which the run function will return (f(a,b),s)
def map2[B, C](sb: State[S, B])(f: (A, B) => C): State[S, C] =
flatMap(a => sb.map(b => f(a, b)))
def flatMap[B](f: A => State[S, B]): State[S, B] =
State(s => {
val (a, s1) = run(s)
val sB = f(a)
sB.run(s1)
})
}
I am new to functional programming and I am dizzied by this code, I tried to understand what does flatMap really does but it looks very abstract to me that I couldn't understand it. For example here we defined map in terms of flatMap, but I can't understand how can we explain the implementation I mean how does it work?
This chapter is about pure functional state, and describes how the state is being updated in a sequence of operations in a functional way, that is to say, by means of functional composition. So you don´t have to create a global state variable that is mutated by a sequence of method executions, as you probably would do in a OOP fashion.
Having said that, this wonderful book explains each concept declaring an abstraction to represent an effect, and some primitives which are going to be used to build more complex functions. In this chapter, the effect is the State. It is represented as a function that takes a previous state and yields a value and a new state, as you can see with the run function declaration. So if you want to compose functions that propagate some state in a pure functional fashion, all of them need to return a new functional State.
With flatMap, which is a primitive of the State type, you can compose functions and, at the same time, propagate some internal state.
For example, if you have two functions:
case class Intermediate(v: Double = 0)
def execute1(a: Int): State[Int, Intermediate] = {
State { s =>
/** Do whatever */
(Intermediate(a * a), s + 1)
}
}
def execute2(a: Int): State[Int, Intermediate] = {
State { s =>
/** Do whatever */
(Intermediate(a * a), s + 1)
}
}
As you can see each function consist in a State function that takes a previous state, executes some logic, and returns a tuple of Int, which is going to be used to track the number of functions that have been executed, and a Intermediate value that we use to yield the execution of every function.
Now, if you apply flat map to execute both functions sequentially:
val result: State[Int, Intermediate] =
execute1(2) flatMap (r => execute2(r.v.toInt))
the result val is another State that you can see as the point of entry to run the pipeline:
/** 0 would be the initial state **/
result.run(0)
the result is:
(Intermediate(16.0),2)
How this works under the hood:
First you have the first function execute1. This function returns a State[Intermediate, Int]. I will use Intermediate to store the result of each execution, that would be the yielded value, and an Int to count the number of functions that are going to be executed in the flatMap sequential composition.
Then I apply flatMap after the function call, which is the flatMap of the State type.
Remember the definition:
def flatMap[B](f: A => State[S, B]): State[S, B] =
State(s => {
/* This function is executed in a State context */
val (a, s1) = run(s)
val sB = f(a)
sB.run(s1)
})
The execution is inside a state context(note that the flatMap function is inside the State type) which is be the first state function of the execute1 function. Maybe this representation is easier:
val st1 = execute1(2)
st1.flatMap(r => execute2(r.v.toInt))
So when run(s) is executed, it actually executes the function state returned by the execute1(2) function. It gives the first (a, s1) result.
Then you have to execute the second function, execute2. You can visualize this function as the input parameter of the flatMap function. This function is executed with the yielded value of the execute1(2) function (the Intermediate type, left side of the tuple). This execution returns another state function that uses the state(right value of the tuple) of the execution1, to create the final tuple with the resulting yielded value of the execute2 function and the int corresponding to the number of executions. Note, that in each function we increment the internal state in one unit.
Note that the result is another State. This has a run method that waits for the initial state.
I have tried to explain the workflow of function calls... it is a little bit hard if you are not used to think in a functional style, but it is only functional composition, it becomes familiar after a while.
I took the scala odersky course and thought that the function that Flatmap takes as arguments , takes an element of Monad and returns a monad of different type.
trait M[T] {
def flatMap[U](f: T => M[U]): M[U]
}
On Monad M[T] , the return type of function is also the same Monad , the type parameter U might be different.
However I have seen examples on internet , where the function returns a completely different Monad. I was under impression that return type of function should the same Monad. Can someone simplify the below to explain how flapmap results in the actual value instead of Option in the list.
Is the List not a Monad in Scala.
val l= List(1,2,3,4,5)
def f(x:int) = if (x>2) Some(x) else None
l.map(x=>f(x))
//Result List[Option[Int]] = List( None , None , Some(3) , Some(4) , Some(5))
l.flatMap(x=>f(x))
//Result: List(3,4,5)
Let's start from the fact that M[T]is not a monad by itself. It's a type constructor. It becomes a monad when it's associated with two operators: bind and return (or unit). There are also monad laws these operators must satisfy, but let's omit them for brevity. In Haskell the type of bind is:
class Monad m where
...
(>>=) :: m a -> (a -> m b) -> m b
where m is a type constructor. Since Scala is OO language bind will look like (first argument is self):
trait M[T] {
def bind[U](f: T => M[U]): M[U]
}
Here M === m, T === a, U === b. bind is often called flatMap. In a pure spherical world in a vacuum that would be a signature of flatMap in OO language. Scala is a very practical language, so the real signature of flatMap for List is:
final def flatMap[B, That](f: (A) ⇒ GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
It's not bind, but will work as a monadic bind if you provide f in the form of (A) => List[B] and also make sure that That is List[B]. On the other hand Scala is not going to watch your back if you provide something different, but will try to find some meaningful conversion (e.g. CanBuildFrom or something else) if it exists.
UPDATE
You can play with scalac flags (-Xlog-implicits, -Xlog-implicit-conversions) to see what's happening:
scala> List(1).flatMap { x => Some(x) }
<console>:1: inferred view from Some[Int] to scala.collection.GenTraversableOnce[?] via scala.this.Option.option2Iterable[Int]: (xo: Option[Int])Iterable[Int]
List(1).flatMap { x => Some(x) }
^
res1: List[Int] = List(1)
Hmm, perhaps confusingly, the signature you gave is not actually correct, since it's really (in simplified form):
def flatMap[B](f: (A) ⇒ GenTraversableOnce[B]): Traversable[B]
Since the compiler is open-source, you can actually see what it's doing (with its full signature):
def flatMap[B, That](f: A => GenTraversableOnce[B])(implicit bf: CanBuildFrom[Repr, B, That]): That = {
def builder = bf(repr) // extracted to keep method size under 35 bytes, so that it can be JIT-inlined
val b = builder
for (x <- this) b ++= f(x).seq
b.result
}
So you can see that there is actually no requirement that the return type of f be the same as the return type of flatMap.
The flatmap found in the standard library is a much more general and flexible method than the monadic bind method like the flatMap from Odersky's example.
For example, the full signature of flatmap on List is
def flatMap[B, That](f: (A) ⇒ GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
Instead of requiring the function passed into flatmap to return a List, it is able to return any GenTraversableOnce object, a very generic type.
flatmap then uses the implicit CanBuildFrom mechanism to determine the appropriate type to return.
So when you use flatmap with a function that returns a List, it is a monadic bind operation, but it lets you use other types as well.
I have been trying to understand the State Monad. Not so much how it is used, though that is not always easy to find, either. But every discussion I find of the State Monad has basically the same information and there is always something I don't understand.
Take this post, for example. In it the author has the following:
case class State[S, A](run: S => (A, S)) {
...
def flatMap[B](f: A => State[S, B]): State[S, B] =
State(s => {
val (a, t) = run(s)
f(a) run t
})
...
}
I can see that the types line up correctly. However, I don't understand the second run at all.
Perhaps I am looking at the whole purpose of this monad incorrectly. I got the impression from the HaskellWiki that the State monad was kind of like a state-machine with the run allowing for transitions (though, in this case, the state-machine doesn't really have fixed state transitions like most state machines). If that is the case then in the above code (a, t) would represent a single transition. The application of f would represent a modification of that value and State (generating a new State object). That leaves me completely confused as to what the second run is all about. It would appear to be a second 'transition'. But that doesn't make any sense to me.
I can see that calling run on the resulting State object produces a new (A, S) pair which, of course, is required for the types to line up. But I don't really see what this is supposed to be doing.
So, what is really going on here? What is the concept being modeled here?
Edit: 12/22/2015
So, it appears I am not expressing my issue very well. Let me try this.
In the same blog post we see the following code for map:
def map[B](f: A => B): State[S, B] =
State(s => {
val (a, t) = run(s)
(f(a), t)
})
Obviously there is only a single call to run here.
The model I have been trying to reconcile is that a call to run moves the state we are keeping forward by a single state-change. This seems to be the case in map. However, in flatMap we have two calls to run. If my model was correct that would result in 'skipping over' a state change.
To make use of the example #Filppo provided below, the first call to run would result in returning (1, List(2,3,4,5)) and the second would result in (2, List(3,4,5)), effectively skipping over the first one. Since, in his example, this was followed immediately by a call to map, this would have resulted in (Map(a->2, b->3), List(4,5)).
Apparently that is not what is happening. So my whole model is incorrect. What is the correct way to reason about this?
2nd Edit: 12/22/2015
I just tried doing what I said in the REPL. And my instincts were correct which leaves me even more confused.
scala> val v = State(head[Int]).flatMap { a => State(head[Int]) }
v: State[List[Int],Int] = State(<function1>
scala> v.run(List(1,2,3,4,5))
res2: (Int, List[Int]) = (2,List(3, 4, 5))
So, this implementation of flatMap does skip over a state. Yet when I run #Filippo's example I get the same answer he does. What is really happening here?
To understand the "second run" let's analyse it "backwards".
The signature def flatMap[B](f: A => State[S, B]): State[S, B] suggests that we need to run a function f and return its result.
To execute function f we need to give it an A. Where do we get one?
Well, we have run that can give us A out of S, so we need an S.
Because of that we do: s => val (a, t) = run(s) ....
We read it as "given an S execute the run function which produces us A and a new S. And this is our "first" run.
Now we have an A and we can execute f. That's what we wanted and f(a) gives us a new State[S, B].
If we do that then we have a function which takes S and returns Stats[S, B]:
(s: S) =>
val (a, t) = run(s)
f(a) //State[S, B]
But function S => State[S, B] isn't what we want to return! We want to return just State[S, B].
How do we do that? We can wrap this function into State:
State(s => ... f(a))
But it doesn't work because State takes S => (B, S), not S => State[B, S].
So we need to get (B, S) out of State[B, S].
We do it by just calling its run method and providing it with the state we just produced on the previous step!
And it is our "second" run.
So as a result we have the following transformation performed by a flatMap:
s => // when a state is provided
val (a, t) = run(s) // produce an `A` and a new state value
val resState = f(a) // produce a new `State[S, B]`
resState.run(t) // return `(S, B)`
This gives us S => (S, B) and we just wrap it with the State constructor.
Another way of looking at these "two runs" is:
first - we transform the state ourselves with "our" run function
second - we pass that transformed state to the function f and let it do its own transformation.
So we kind of "chaining" state transformations one after another. And that's exactly what monads do: they provide us with the ability to schedule computation sequentially.
The state monad boils down to this function from one state to another state (plus A):
type StatefulComputation[S, +A] = S => (A, S)
The implementation mentioned by Tony in that blog post "capture" that function into run of the case class:
case class State[S, A](run: S => (A, S))
The flatmap implementation to bind a state to another state is calling 2 different runs:
// the `run` on the actual `state`
val (a: A, nextState: S) = run(s)
// the `run` on the bound `state`
f(a).run(nextState)
EDIT Example of flatmap between 2 State
Considering a function that simply call .head to a List to get A, and .tail for the next state S
// stateful computation: `S => (A, S)` where `S` is `List[A]`
def head[A](xs: List[A]): (A, List[A]) = (xs.head, xs.tail)
A simple binding of 2 State(head[Int]):
// flatmap example
val result = for {
a <- State(head[Int])
b <- State(head[Int])
} yield Map('a' -> a,
'b' -> b)
The expect behaviour of the for-comprehension is to "extract" the first element of a list into a and the second one in b. The resulting state S would be the remaining tail of the run list:
scala> result.run(List(1, 2, 3, 4, 5))
(Map(a -> 1, b -> 2),List(3, 4, 5))
How? Calling the "stateful computation" head[Int] that is in run on some state s:
s => run(s)
That gives the head (A) and the tail (B) of the list. Now we need to pass the tail to the next State(head[Int])
f(a).run(t)
Where f is in the flatmap signature:
def flatMap[B](f: A => State[S, B]): State[S, B]
Maybe to better understand what is f in this example, we should de-sugar the for-comprehension to:
val result = State(head[Int]).flatMap {
a => State(head[Int]).map {
b => Map('a' -> a, 'b' -> b)
}
}
With f(a) we pass a into the function and with run(t) we pass the modified state.
I have accepted #AlexyRaga's answer to my question. I think #Filippo's answer was very good as well and, in fact, gave me some additional food for thought. Thanks to both of you.
I think the conceptual difficulty I was having was really mostly to do with 'what does the run method 'mean'. That is, what is its purpose and result. I was looking at it as a 'transition' function (from one state to the next). And, after a fashion, that is what it does. However, it doesn't transition from a given (this) state to the next state. Instead, it takes an initial State and returns the (this) state's value and a new 'current' state (not the next state in the state-transition sequence).
That is why the flatMap method is implemented the way it is. When you generate a new State then you need the current value/state pair from it based on the passed-in initial state which can then be wrapped in a new State object as a function. You are not really transitioning to a new state. Just re-wrapping the generated state in a new State object.
I was too steeped in traditional state machines to see what was going on here.
Thank, again, everyone.
How does immutable.Map.map work? It looks like there is something wrong with the documentation:
def map[B](f: (A) ⇒ B): Map[B]
[use case]
Builds a new collection by applying a function to all elements of this immutable map.
Full Signature
def map[B, That](f: ((A, B)) ⇒ B)(implicit bf: CanBuildFrom[Map[A, B], B, That]): That
Map[B] does not make sense since Map takes two type parameters.
In the full signature, there is a name conflict between B the type argument of map, and B the type parameter of Map.
The understandable confusion stems from the fact that map is not implemented in Map, but in TraversableLike. However, the function documentation in inherited by sub classes.
TraversableLike takes two type parameters TraversableLike[+A, +Repr] and the map function has the signature TraversableLike.map[B](f: (A) ⇒ B): Traversable[B]. In the api docs of Map the documentation for that is inherited and partly adjusted Traversable[B] by Map[B], but B from TraversableLike is not resolved to (A, B) form Map.
(Might be a bug in Scaladoc, that probably won't be fixed, as there could be an erasure problem. But that's just guessing on my side.)
You can check what is actually implemented in Map if you configure visibility of members right above the members documentation.
EDIT:
Now to your core question:
If the documentation would give us, what we could read intuitively
and if we simplify it a bit for readability
and if we than further use a bit more of natural language instead of acronyms, the signature of map for a Map[A, B] could look like:
map[ResultItem, ResultCollection](f: (A,B) => ResultItem)(implicit bf: CanBuildFrom[Map[A, B], ResultItem, ResultCollection]): ResultCollection
So, basically you apply a function to each key-value-pair of the Map that transforms a key-value-pair of type (A,B) into a value of type ResultType.
As you can build almost any kind of collection from a map (or any other collection), this result type does not have to be another tuple. And ResultCollection does not have to be another Map. E.g.:
Map("1" -> 1, "2" -> 2).map((keyValuePair: (String, Int)) => keyValuePair._2)
or short
Map("1" -> 1, "2" -> 2).map(_._2)
has List(1, 2) as result, List[Int] as ResultCollection, Int as ResultItem.
This is possible because of the implicit CanBuildFrom parameter that adds an implicit builder that takes the result of the map function and appends it to its builder-result. In most cases CanBuildFrom is inferred by the compiler. However, there are cases when it will not be able to infer the proper result collection.
In such cases you have to give the compiler more information:
val test2: Vector[Int] = Map("1" -> 1, "2" -> 2).map(_._2)(collection.breakOut)
val test3: Set[Int] = Map("1" -> 1, "2" -> 2).map(_._2)(collection.breakOut)
For more information on breakOut and CanBuildFrom I'd recommend this answer
I'm working with this code:
case class State[S, +A](run: S => (A, S)) {
...
def flatMap[B](f: A => State[S, B]): State[S, B] = State(s => {
val (a, s1) = run(s)
f(a).run(s1)
})
...
}
This is an abstraction for working with purely functional state, from §6 of FP in Scala. run is function parameter that takes a state and emits a tuple of a value and the new state.
My question is around the syntax s => in this section:
... B] = State(s => { ...
This appears to be using the State 'constructor' (ie apply) to build a new State object. But what does the s represent? Is it an 'anonymous' State, representing any State instance? If so, how is it different from this? Or does s correspond to the input parameter of run ie the S from:
... (run: S => ....
And why would I be using a constructor for a function definition? Notice that the last symbol of the flatMap definition is a ) not a }, that is closing the State apply constructor.
This scenario is a bit different than the standard
case class Person(name: String)
scenario, so I thought I'd ask...
Your second assumption is correct, s corresponds to the input parameter of run function, ie the S, which represents an actual state passing through the chain. So s => {...} is just a lambda definition of type S => (A, S). Lambdas and functions in Scala are first-class citizens (values), so you can pass them as parameters to hold inside some another type (including some monad).
Here the function, which produces new raw state S (and new result A), is wrapped (see return operation) to the monad State, which is implemented as case class. We need it to define flatMap operation (see bind) over the monad.
To make it more clear about the passing function as parameter, the code may be rewritten to:
case class State[S, +A](run: S => (A, S)) {
def flatMap[B](f: A => State[S, B]): State[S, B] = {
def newState(s: S) = {
val (a, s1) = run(s)
f(a).run(s1)
}
State(newState _)
}
}
So, according to the monad definition:
State[S, +A] is a type constructor which takes two plain types (S and covariant A) and returns monadic type State
State.apply(run: S => (A, S)) function takes a plain function and returns (lifts into) monadic container State, so it is a "return" operator of the monad
State.flatMap[B](f: A => State[S, B]): State[S, B] is corresponding to the "bind" operator
Case class is used just to have explicit "return" function (apply) instead of using new operator.