For a coding assignment for school I have to do stuff with flatmap but I have absolutely no idea what it does and I've read a couple pages online and read in my textbook but I still have no true understanding of what it does. I know what map does but for some reason it's hard for me to wrap my head around flatmap. Can anyone help? Thanks.
Just to add some more information- when I look at examples online I DO see how flatmap returns something different from map. But what is flatmap actually doing when its being called on something? How does flatmap actually work? What is it doing before it returns the result?
Here's an analogy.
Imagine you have a big bag filled with shopping vouchers for cartons of eggs. If you have a function which is "use the voucher to buy a carton of eggs" and you called bigBagOfVouchers.map(buyCartonOfEggs), you'd have a bag of cartons of eggs.
However, if you called bigBagOfVouchers.flatMap(buyCartonOfEggs), you'd have a bag of eggs - without any cartons.
flatMap flattens the result by one level. What might have been Bag[Carton[Egg]] is now Bag[Egg].
Functors define map which have type
trait Functor[F[_]] {
def map[A, B](f: A => B)(v: F[A]): F[B]
}
Monads are functors which support two additional operations:
trait Monad[M[_]] extends Functor[M] {
def pure[A](v: A): M[A]
def join[A](m: M[M[A]]): M[A]
}
Join flattens nested values e.g. if m is List then join has type
def joinList[A](l: List[List[A]]): List[A]
If you have a monad m and you map over it, what happens if b is the same monadic type? For example:
def replicate[A](i: Int, value: A): List[A] = ???
val f = new Functor[List] {
def map[A, B](f: A => B)(v: List[A]) = v.map(f)
}
then
f.map(x => replicate(x, x))(List(1,2,3)) == List(List(1), List(2,2), List(3,3,3))
This has type List[List[Int]] while the input is a List[Int]. It's fairly common with a chain of operations to want each step to return the same input type. Since List can also be made into a monad, you can easily create such a list using join:
listMonad.join(List(List(1), List(2,2), List(3,3,3))) == List(1,2,2,3,3,3)
Now you might want to write a function to combine these two operations into one:
trait Monad[M] {
def flatMap[A, B](f: A => M[B])(m: M[A]): M[B] = join(map(f)(m))
}
then you can simply do:
listMonad.flatMap(List(1,2,3), x => replicate(x, x)) == List(1,2,2,3,3,3)
Exactly what flatMap does depends on the monad type constructor M (List in this example) since it depends on map and join.
In reactive programming, you often come into the situation in which you need to use flatMap to convert Future[Future[List]] to Future[List]. For example, you have two functions: get Users from database and process retrieved Users; and both return Future[List[User]]. If you apply map to get and process, the result will be Future[Future[List[User]]] which does not make any sense. Instead, you should use flatMap:
def main(): Future[List[User]] = getUsers flatMap processUsers
def getUsers: Future[List[User]]
def processUsers(users: List[User]): Future[List[User]]
Related
flatMap signature:
/* applies a transformation of the monad "content" by composing
* this monad with an operation resulting in another monad instance
* of the same type
*/
def flatMap(f: A => M[B]): M[B]
is there anyway to understand by its signature (input to output) (except for the name flat) that its flattening the structure? or must I read its implementation to understand that? isn't good coding practice means I can understand by the signature of the function (input to output even without function name) exactly what it does? if so how does flatMap follows this basic programming practice? or does it violates it?
It helps to understand that the type of the thing that flatMap is defined on is also M and would be something like Traversable[A] or a subclass such as a List[A], or another Monad such as Option or Future. Therefore the left hand side of the function definition that flatMap requires (f: A => M[B]) hints to us that this function processes the individual items contained within the Monad, and produces M[B] for each one. The fact that it then returns M[B] rather than M[M[B]] is a hint that some flattening is taking place.
However I disagree that it should be possible to know exactly what a function does by its signature. For example:
Trait Number {
def plus(that: Number): Number
def minus(that: Number): Number
}
Without the function names and possible also some documentation, I don't think it is reasonable to expect that another person can know what plus and minus do.
Does flatMap functional signature (input -> output) proposes its doing any flattening?
Yes, it does.
In fact, join (or flatten), map and unit form a minimal set of primitive functions needed to implement a monad.
flatMap can be implemented in terms of these other functions.
//minimal set
def join[A](mma: M[M[A]]): M[A] = ???
def map[A](ma: M[A], f: A => B): M[B] = ???
def unit[A](a: A): M[A] = ???
def flatMap[A](ma: M[A], f: A => M[B]): M[B] = join(map(ma))
It should now be clear that flatMap flattens the structure of the mapped monad.
For comparison, here's another mimimal set of primitives, where join/flatten is implemented in terms of flatMap.
// minimal set
def unit[A](a: A): M[A] = ???
def flatMap[A](ma: M[A], f: A => M[B]): M[B] = ???
def map[A](ma: M[A], f: A => B): M[B] = flatMap(ma, a => unit(f(a)))
def join[A](mma: M[M[A]]): M[A] = flatMap(mma, ma => ma)
I'm working on this Functional Programming in Scala exercise:
// But what if our list has an element type that doesn't have a Monoid instance?
// Well, we can always map over the list to turn it into a type that does.
As I understand this exercise, it means that, if we have a Monoid of type B, but our input List is of type A, then we need to convert the List[A] to List[B], and then call foldLeft.
def foldMap[A, B](as: List[A], m: Monoid[B])(f: A => B): B = {
val bs = as.map(f)
bs.foldLeft(m.zero)((s, i) => m.op(s, i))
}
Does this understanding and code look right?
First I'd simplify the syntax of the body a bit:
def foldMap[A, B](as: List[A], m: Monoid[B])(f: A => B): B =
as.map(f).foldLeft(m.zero)(m.ops)
Then I'd move the monoid instance into its own implicit parameter list:
def foldMap[A, B](as: List[A])(f: A => B)(implicit m: Monoid[B]): B =
as.map(f).foldLeft(m.zero)(m.ops)
See the original "Type Classes as Objects and Implicits" paper for more detail about how Scala implements type classes using implicit parameter resolution, or this answer by Rex Kerr that I've also linked above.
Next I'd switch the order of the other two parameter lists:
def foldMap[A, B](f: A => B)(as: List[A])(implicit m: Monoid[B]): B =
as.map(f).foldLeft(m.zero)(m.ops)
In general you want to place the parameter lists containing parameters that change the least often first, in order to make partial application more useful. In this case there may only be one possible useful value of A => B for any A and B, but there are lots of values of List[A].
For example, switching the order allows us to write the following (which assumes a monoid instance for Bar):
val fooSum: List[Foo] => Bar = foldMap(fooToBar)
Finally, as a performance optimization (mentioned by stew above), you could avoid creating an intermediate list by moving the application of f into the fold:
def foldMap[A, B](f: A => B)(as: List[A])(implicit m: Monoid[B]): B =
as.foldLeft(m.zero) {
case (acc, a) => m.op(acc, f(a))
}
This is equivalent and more efficient, but to my eye much less clear, so I'd suggest treating it like any optimization—if you need it, use it, but think twice about whether the gains are really worth the loss of clarity.
My problem is phrased in the code below.
I'm trying to get some input that has the .map function in it. I know that if I call .map to it, it will return an Int to me.
// In my case, they are different representations of Ints
// By that I mean that in the end it all boils down to Int
val list: Seq[Int] = Seq(1,2,3,4)
val optInt: Option[Int] = Some(1)
// I can use a .map with a Seq, check!
list.map {
value => println(value)
}
// I can use it with an Option, check!
optInt.map {
value => println(value)
}
// Well, you're asking yourself why do I have to do it,
// Why don't I use foreach to solve my problem. Check!
list.foreach(println)
optInt.foreach(println)
// The problem is that I don't know what I'm going to get as input
// The only thing I know is that it's "mappable" (it has the .map function)
// And that if I were to apply .map it would return Ints to me
// Like this:
def printValues(genericInputThatHasMap: ???) {
genericInputThatHasMap.map {
value => println(value)
}
}
// The point is, what do I have to do to have this functionality?
// I'm researching right now, but I still haven't found anything.
// That's why I'm asking it here =(
// this works:
def printValues(genericInputThatHasMap: Seq[Int]) {
genericInputThatHasMap.map {
value => println(value)
}
}
Thanks in advance! Cheers!
First for a quick note about map and foreach. If you're only interested in performing an operation with a side effect (e.g., printing to standard output or a file, etc.) on each item in your collection, use foreach. If you're interested in creating a new collection by transforming each element in your old one, use map. When you write xs.map(println), you will in fact print all the elements of the collection, but you'll also get back a (completely useless) collection of units, and will also potentially confuse future readers of your code—including yourself—who expect foreach to be used in a situation like this.
Now on to your problem. You've run into what is in my opinion one of the ugliest warts of the Scala standard library—the fact that methods named map and foreach (and flatMap) get magical treatment at the language level that has nothing to do with a specific type that defines them. For example, I can write this:
case class Foo(n: Int) {
def foreach(f: Int => Unit) {
(0 until n) foreach f
}
}
And use it in a for loop like this, simply because I've named my method foreach:
for (i <- Foo(10)) println(i)
You can use structural types to do something similar in your own code:
def printValues(xs: { def foreach(f: (Int) => Unit): Unit }) {
xs foreach println
}
Here any xs with an appropriately typed foreach method—for example an Option[Int] or a List[Int]—will compile and work as expected.
Structural types get a lot messier when you're trying to work with map or flatMap though, and are unsatisfying in other ways—they impose some ugly overhead due to their use of runtime reflection, for example. They actually have to be explicitly enabled in Scala 2.10 to avoid warnings for these reasons.
As senia's answer points out, the Scalaz library provides a much more coherent approach to the problem through the use of type classes like Monad. You wouldn't want to use Monad, though, in a case like this: it's a much more powerful abstraction than you need. You'd use Each to provide foreach, and Functor for map. For example, in Scalaz 7:
import scalaz._, Scalaz._
def printValues[F[_]: Each](xs: F[Int]) = xs foreach println
Or:
def incremented[F[_]: Functor](xs: F[Int]) = xs map (_ + 1)
To summarize, you can do what you want in a standard, idiomatic, but arguably ugly way with structural types, or you can use Scalaz to get a cleaner solution, but at the cost of a new dependency.
My thoughts on the two approaches.
Structural Types
You can use a structural type for foreach, but for map it doesn't appear you can construct one to work across multiple types. For example:
import collection.generic.CanBuildFrom
object StructuralMap extends App {
type HasMapAndForeach[A] = {
// def map[B, That](f: (A) ⇒ B)(implicit bf: CanBuildFrom[List[A], B, That]): That
def foreach[B](f: (A) ⇒ B): Unit
}
def printValues(xs: HasMapAndForeach[Any]) {
xs.foreach(println _)
}
// def mapValues(xs: HasMapAndForeach[Any]) {
// xs.map(_.toString).foreach(println _)
// }
def forComp1(xs: HasMapAndForeach[Any]) {
for (i <- Seq(1,2,3)) println(i)
}
printValues(List(1,2,3))
printValues(Some(1))
printValues(Seq(1,2,3))
// mapValues(List(1,2,3))
}
scala> StructuralMap.main(new Array[String](0))
1
2
3
4
5
6
7
8
9
10
See the map method commented out above, it has List hardcoded as a type parameter in the CanBuildFrom implicit. There might be a way to pick up the type generically - I will leave that as a question to the Scala type gurus out there. I tried substituting HasMapAndForeach and this.type for List but neither of those worked.
The usual performance caveats about structural types apply.
Scalaz
Since structural types is a dead end if you want to support map then let's look at the scalaz approach from Travis and see how it works. Here are his methods:
def printValues[F[_]: Each](xs: F[Int]) = xs foreach println
def incremented[F[_]: Functor](xs: F[Int]) = xs map (_ + 1)
(In the below correct me if I am wrong, I am using this as a scalaz learning experience)
The typeclasses Each and Functor are used to restrict the types of F to ones where implicits are available for Each[F] or Functor[F], respectively. For example, in the call
printValues(List(1,2,3))
the compiler will look for an implicit that satisfies Each[List]. The Each trait is
trait Each[-E[_]] {
def each[A](e: E[A], f: A => Unit): Unit
}
In the Each object there is an implicit for Each[TraversableOnce] (List is a subtype of TraversableOnce and the trait is contravariant):
object Each {
implicit def TraversableOnceEach[A]: Each[TraversableOnce] = new Each[TraversableOnce] {
def each[A](e: TraversableOnce[A], f: A => Unit) = e foreach f
}
}
Note that the "context bound" syntax
def printValues[F[_]: Each](xs: F[Int])
is shorthand for
def printValues(xs: F[Int])(implicit ev: Each[F])
Both of these denote that F is a member of the Each typeclass. The implicit that satisfies the typeclass is passed as the ev parameter to the printValues method.
Inside the printValues or incremented methods the compiler doesn't know that xs has a map or foreach method because the type parameter F doesn't have any upper or lower bounds. As far as it can tell F is AnyRef and satisfies the context bound (is part of the typeclass). What is in scope that does have foreach or map? MA from scalaz has both foreach and map methods:
trait MA[M[_], A] {
def foreach(f: A => Unit)(implicit e: Each[M]): Unit = e.each(value, f)
def map[B](f: A => B)(implicit t: Functor[M]): M[B] = t.fmap(value, f)
}
Note that the foreach and map methods on MA are constrained by the Each or Functor typeclass. These are the same constraints from the original methods so the constraints are satisfied and an implicit conversion to MA[F, Int] takes place via the maImplicit method:
trait MAsLow extends MABLow {
implicit def maImplicit[M[_], A](a: M[A]): MA[M, A] = new MA[M, A] {
val value = a
}
}
The type F in the original method becomes type M in MA.
The implicit parameter that was passed into the original call is then passed as the implicit parameter into foreach or map. In the case of foreach, each is called on its implicit parameter e. In the example from above the implicit ev was type Each[TraversableOnce] because the original parameter was a List, so e is the same type. foreach calls each on e which delegates to foreach on TraversableOnce.
So the order of calls for printValues(List(1,2,3)) is:
new Each[TraversableOnce] -> printValues -> new MA -> MA.foreach -> Each.each -> TraversableOnce.foreach
As they say, there is no problem that can't be solved with an extra level of indirection :)
You can use MA from scalaz:
import scalaz._
import Scalaz._
def printValues[A, M[_]](ma: MA[M, A])(implicit e: Each[M]) {
ma |>| { println _ }
}
scala> printValues(List(1, 2, 3))
1
2
3
scala> printValues(Some(1))
1
an example use case:
def div2(i: Int): Validation[String, Int] =
if (i%2 == 0) Validation.success(i/2)
else Validation.failure("odd")
def div4(i: Int) = for {
a <- div2(i)
b <- div2(a)
} yield b
error: Unable to unapply type scalaz.Validation[String,Int] into a type constructor of kind M[_] that is classified by the type class scalaz.Bind
I guess the error is caused by the compiler can't find a Monad instance for Validation[String, Int]
I can make one for myself, like:
object Instances {
implicit def validationMonad[E] = new Monad[({type L[A] = Validation[E, A]})#L] {
override def point[A](a: => A) =
Validation.success(a)
override def bind[A, B](fa: Validation[E, A])(f: A => Validation[E, B]) =
fa bind f
}
}
but why doesn't Validation have it already? after all, Validation already has the bind method defined.
moreover, I can't have import Validation._ and import Instances._ together anymore (this took me looong to figure out...), because of another complicated error...
ambiguous implicit values: something like both validationMonad (my instance), and method ValidationInstances1 in trait ValidationInstances2... both match some Functor of Validation...
should I modify the source of scalaz? or I'm completely missing something~?
please help~
I'm using scalaz 7.0.0-M2
As discussed in the Scalaz group, the problem seems to be that ap would accumulate errors whereas (pseudo-)monadic composition would only operate on the value part of Validation.
Therefore, one cannot be expressed in terms of the other and thus no monad instance exists for Validation.
The issue is that the applicative functor as implied by the monad does not equal the actual applicative functor
Can please someone explain the differences between Functor and Monad in the Scala context?
Scala itself really does not emphasize the Functor and Monad terms that much. I guess using map is the functor side, using flatMap is the Monad side.
For me looking and playing around with scalaz has been so far the best avenue to get a sense of those functional concepts in the scala context (versus the haskell context). Two years ago when I started scala, the scalaz code was gibberish to me, then a few months ago I started looking again and I realized that it's really a clean implementation of that particular style of functional programming.
For instance the Monad implementation shows that a monad is a pointed functor because it extends the Pointed trait (as well as the Applicative trait). I invite you to go look at the code. It has linking in the source itself and it's really easy to follow the links.
So functors are more general. Monads provide additional features. To get a sense of what you can do when you have a functor or when you have a monad, you can look at MA
You'll see utility methods that need an implicit functor (in particular applicative functors) such as sequence and sometime methods that needs a full monad such as replicateM.
Taking scalaz as the reference point, a type F[_] (that is, a type F which is parameterized by some single type) is a functor if a function can be lifted into it. What does this mean:
class Function1W[A, B](self: A => B) {
def lift[F[_]: Functor]: F[A] => F[B]
}
That is, if I have a function A => B, a functor F[_], then I now have a function F[A] => F[B]. This is really just the reverse-way of looking at scala's map method, which (ignoring the CanBuildFrom stuff) is basically:
F[A] => (A => B) => F[B]
If I have a List of Strings, a function from String to Int, then I can obviously produce a List of Ints. This goes for Option, Stream etc. They are all functors
What I find interesting about this is that you might immediately jump to the (incorrect) conclusion that a Functor is a "container" of As. This is an unnecesssary restriction. For example, think about a function X => A. If I have a function X => A and a function A => B then clearly, by composition, I have a function X => B. But now, look at it this way:
type F[Y] = X => Y //F is fixed in X
(X => A) andThen (A => B) is X => B
F[A] A => B F[B]
So the type X => A for some fixed X is also a functor. In scalaz, functor is designed as a trait as follows:
trait Functor[F[_]] { def fmap[A, B](fa: F[A], f: A => B): F[B] }
hence the Function1.lift method above is implemented
def lift[F[_]: Functor]: F[A] => F[B]
= (f: F[A]) => implicitly[Functor[F]].fmap(f, self)
A couple of functor instances:
implicit val OptionFunctor = new Functor[Option] {
def fmap[A, B](fa: Option[A], f: A => B) = fa map f
}
implicit def Functor1Functor[X] = new Functor[({type l[a]=X => a})#l] {
def fmap[A, B](fa: X => B, f: A => B) = f compose fa
}
In scalaz, a monad is designed like this:
trait Monad[M[_]] {
def pure[A](a: A): M[A] //given a value, you can lift it into the monad
def bind[A, B](ma: M[A], f: A => B): M[B]
}
It is not particularly obvious what the usefulness of this might be. It turns out that the answer is "very". I found Daniel Spiewak's Monads are not Metaphors extremely clear in describing why this might be and also Tony Morris's stuff on configuration via the reader monad, a good practical example of what might be meant by writing your program inside a monad.
A while ago I wrote about that: http://gabrielsw.blogspot.com/2011/08/functors-applicative-functors-and.html (I'm no expert though)
The first thing to understand is the type ' T[X] ' : It's a kind of "context" (is useful to encode things in types and with this you're "composing" them) But see the other answers :)
Ok, now you have your types inside a context, say M[A] (A "inside" M), and you have a plain function f:A=>B ... you can't just go ahead and apply it, because the function expects A and you have M[A]. You need some way to "unpack" the content of M, apply the function and "pack" it again. If you have "intimate" knowledge of the internals of M you can do it, if you generalize it in a trait you end with
trait Functor[T[_]]{
def fmap[A,B](f:A=>B)(ta:T[A]):T[B]
}
And that's exactly what a functor is. It transforms a T[A] into a T[B] by applying the function f.
A Monad is a mythical creature with elusive understanding and multiple metaphors, but I found it pretty easy to understand once you get the applicative functor:
Functor allow us to apply functions to things in a context. But what if the functions we want to apply are already in a context? (And is pretty easy to end in that situation if you have functions that take more than one parameter).
Now we need something like a Functor but that also takes functions already in the context and applies them to elements in the context. And that's what the applicative functor is. Here is the signature:
trait Applicative[T[_]] extends Functor[T]{
def pure[A](a:A):T[A]
def <*>[A,B](tf:T[A=>B])(ta:T[A]):T[B]
}
So far so good.
Now comes the monads: what if now you have a function that puts things in the context? It's signature will be g:X=>M[X] ... you can't use a functor because it expects X=>Y so we'll end with M[M[X]], you can't use the applicative functor because is expecting the function already in the context M[X=>Y] .
So we use a monad, that takes a function X=>M[X] and something already in the context M[A] and applies the function to what's inside the context, packing the result in only one context. The signature is:
trait Monad[M[_]] extends Applicative[M]{
def >>=[A,B](ma:M[A])(f:A=>M[B]):M[B]
}
It can be pretty abstract, but if you think on how to work with "Option" it shows you how to compose functions X=>Option[X]
EDIT: Forgot the important thing to tie it: the >>= symbol is called bind and is flatMap in Scala. (Also, as a side note, there are some laws that functors, applicatives, and monads have to follow to work properly).
The best article laying out in details those two notions is "The Essence of the Iterator Pattern " from Eric Torreborre's Blog.
Functor
trait Functor[F[_]] {
def fmap[A, B](f: A => B): F[A] => F[B]
}
One way of interpreting a Functor is to describe it as a computation of values of type A.
For example:
List[A] is a computation returning several values of type A (non-deterministic computation),
Option[A] is for computations that you may or may not have,
Future[A] is a computation of a value of type A that you will get later, and so on.
Another way of picturing it is as some kind of "container" for values of type A.
It is the basic layer from which you define:
PointedFunctor (to create a value of type F[A]) and
Applic (to provide a method applic, being a computed value inside the container F (F[A => B]), to apply to a value F[A]),
Applicative Functor (aggregation of an Applic and a PointedFunctor).
All three elements are used to define a Monad.