I've been trying to learn functional programming in Scala, and I finally managed to understand how to use the for comprehension to work with state:
#!/usr/bin/env scala
case class State[A,S](run: S => (A,S)) {
def map[B](f: A => B): State[B,S] =
State(s => {
val (a, s1) = run(s)
(f(a), s1)
})
def flatMap[B](f: A => State[B,S]): State[B,S] =
State(s => {
val (a,s1) = run(s)
f(a).run(s1)
})
}
val increment = State[Unit,Int] {
x => ((),x+1)
}
val read = State[Int,Int] {
x => (x,x)
}
def prog = for {
_ <- increment
x <- read
_ <- increment
y <- read
} yield (x,y)
val ans = prog.run(0)._1
println(ans)
Although this runs fine, I did not manage to do something similar using a state monad, it is more complicated than, e.g., Option, because it takes an extra type. How do I do something similar to this code with a state monad?
EDIT: Apparently, my question was not clear. I want to run this using a monad trait, like this one, which I took from "Functional programming in Scala":
def stateMonad[S] = new Monad[({type lambda[x] = State[S,x]})#lambda] {
def unit[A](a: => A): State[S,A] = State(s => (a, s))
def flatMap[A,B](st: State[S,A])(f: A => State[S,B]): State[S,B] =
st flatMap f
}
And then perform the computation by instantiating this with something like val M = stateMonad[Int].
After trying around, I managed to get it working. So, I think I'll end up answering my own question. The solution is
trait Monad[M[_]] {
def unit[A](a: => A): M[A]
def flatMap[A,B](ma: M[A])(f: A => M[B]): M[B]
}
class StateMonad[S] extends Monad[({type lambda[x] = State[x,S]})#lambda] {
def unit[A](a: => A): State[A,S] = State(s => (a, s))
def flatMap[A,B](st: State[A,S])(f: A => State[B,S]): State[B,S] =
st flatMap f
def increment: State[Unit,Int] = State(x => ((),x+1))
def read: State[Int,Int] = State(x => (x,x))
}
val m = new StateMonad[Int]
def prog = for {
_ <- m.increment
x <- m.read
_ <- m.increment
y <- m.read
} yield (x,y)
The idea is to make the StateMonad class inherit from Monad, and include all the functions that manipulate state as methods of the StateMonad class. As was pointed out, my previous code could already be considered a monad, but I think doing it this way is better.
Related
I have multiple map functions running over the same data and I'd like to have them run in a single pass. I'm looking for a generic way to do this.
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if(c1 > c2) c1 else c2
def mapG(s: String): Int = s.length
def reduceG(i1: Int, i2: Int): Int = i1 + i2
val largestStartingChar = fruits.map(mapF).reduce(reduceF)
val totalStringLength = fruits.map(mapG).reduce(reduceG)
I'd like to reduce the number of passes over fruits. I can make this generic for two maps and reduces like this:
def productMapFunction[A, B, C](f: A=>B, g: A=>C): A => (B, C) = {
x => (f(x), g(x))
}
def productReduceFunction[T, U](f: (T, T)=>T, g: (U, U) => U):
((T,U), (T,U)) => (T, U) = {
(tu1, tu2) => (f(tu1._1, tu2._1), g(tu1._2, tu2._2))
}
val xMapFG = productMapFunction(mapF, mapG)
val xReduceFG = productReduceFunction(reduceF, reduceG)
val (largestStartingChar2, totalStringLength2) =
fruits.map(xMapFG).reduce(xReduceFG))
I'd like to do this even more generically, with an arbitrary number of map and reduce functions, but I'm not sure how to proceed, or if this is possible.
Following solution uses Cats 2 and custom type MapReduce.
Reducing operation can be specified by function reduce: (O, O) => O
or cats reducer: Semigroup[O].
Multiple MapReduce objects can be combined into one by Apply instance provided by implicit def mapReduceApply[I]
import cats._
import cats.implicits._
trait MapReduce[I, O] {
type R
def reducer: Semigroup[R]
def map: I => R
def mapResult: R => O
def apply(input: Seq[I]): O = mapResult(input.map(map).reduce(reducer.combine))
}
object MapReduce {
def apply[I, O, _R](_reducer: Semigroup[_R], _map: I => _R, _mapResult: _R => O): MapReduce[I, O] =
new MapReduce[I, O] {
override type R = _R
override def reducer = _reducer
override def map = _map
override def mapResult = _mapResult
}
def apply[I, O](map: I => O)(implicit r: Semigroup[O]): MapReduce[I, O] =
MapReduce[I, O, O](r, map, identity)
def apply[I, O](map: I => O, reduce: (O, O) => O): MapReduce[I, O] = {
val reducer = new Semigroup[O] {
override def combine(x: O, y: O): O = reduce(x, y)
}
MapReduce(map)(reducer)
}
implicit def mapReduceApply[I] =
new Apply[({type F[X] = MapReduce[I, X]})#F] {
override def map[A, B](f: MapReduce[I, A])(fn: A => B): MapReduce[I, B] =
MapReduce(f.reducer, f.map, f.mapResult.andThen(fn))
override def ap[A, B](ff: MapReduce[I, (A) => B])(fa: MapReduce[I, A]): MapReduce[I, B] =
MapReduce(ff.reducer product fa.reducer,
i => (ff.map(i), fa.map(i)),
(t: (ff.R, fa.R)) => ff.mapResult(t._1)(fa.mapResult(t._2))
)
}
}
object MultiMapReduce extends App {
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if (c1 > c2) c1 else c2
val biggestFirsChar = MapReduce(mapF, reduceF)
val totalChars = MapReduce[String, Int](_.length) // (Semigroup[Int]) reduce by _ + _
def count[A] = MapReduce[A, Int](_ => 1)
val multiMapReduce = (biggestFirsChar, totalChars, count[String]).mapN((_, _, _))
println(multiMapReduce(fruits))
val sum = MapReduce[Double, Double](identity)
val average = (sum, count[Double]).mapN(_ / _)
println(sum(List(1, 2, 3, 4)))
println(average(List(1, 2, 3, 4)))
}
Runnable version is also available on GitHub.
Interesting question!
I don't know of any such implementation in the standard library or even scalaz/cats.
It's not very surprising because if your list is not very large you can just perform map-reduces sequentially and I'm not even sure that overhead of constructing lots of intermediate objects would be smaller than overhead of traversing the list several times.
And if the list is potentially doesn't fit into memory you should be using one of the streaming libraries (fs2/zio-streams/akka-streams)
Although if your input was Iterator instead of List, such functionality would be useful.
There is an interesting article about this problem:
https://softwaremill.com/beautiful-folds-in-scala/
tldr:
Map-reduce workflow could be formalized as follows:
trait Fold[I, O] {
type M
def m: Monoid[M]
def tally: I => M
def summarize: M => O
}
In your case I = List[A], tally = list => list.map(mapF), summarize = list => list.reduce(reduceF) .
To run a map-reduce on a list using an instance of fold you need to run
fold.summarize(fold.tally(list))
You can define combine operation on them:
def combine[I, O1, O2](f1: Fold[I, O1], f2: Fold[I, O2]): Fold[I, (O1, O2)]
Using combine few times would give you what you want:
combine(combine(f1, f2), f3): Fold[I, ((O1, O2), O3)]
I think you're just trying to reinvent transducers. It's been a while since I have used Scala, but there's at least one implementation.
When I learn State Monad, I'm not sure how to compose two functions with different State return types.
State Monad definition:
case class State[S, A](runState: S => (S, A)) {
def flatMap[B](f: A => State[S, B]): State[S, B] = {
State(s => {
val (s1, a) = runState(s)
val (s2, b) = f(a).runState(s1)
(s2, b)
})
}
def map[B](f: A => B): State[S, B] = {
flatMap(a => {
State(s => (s, f(a)))
})
}
}
Two different State types:
type AppendBang[A] = State[Int, A]
type AddOne[A] = State[String, A]
Two methods with differnt State return types:
def addOne(n: Int): AddOne[Int] = State(s => (s + ".", n + 1))
def appendBang(str: String): AppendBang[String] = State(s => (s + 1, str + " !!!"))
Define a function to use the two functions above:
def myAction(n: Int) = for {
a <- addOne(n)
b <- appendBang(a.toString)
} yield (a, b)
And I hope to use it like this:
println(myAction(1))
The problem is myAction is not compilable, it reports some error like this:
Error:(14, 7) type mismatch;
found : state_monad.State[Int,(Int, String)]
required: state_monad.State[String,?]
b <- appendBang(a.toString)
^
How can I fix it? Do I have to define some Monad transformers?
Update: The question may be not clear, let me give an example
Say I want to define another function, which uses addOne and appendBang internally. Since they all need existing states, I have to pass some to it:
def myAction(n: Int)(addOneState: String, appendBangState: Int): ((String, Int), String) = {
val (addOneState2, n2) = addOne(n).runState(addOneState)
val (appendBangState2, n3) = appendBang(n2.toString).runState(appendBangState)
((addOneState2, appendBangState2), n3)
}
I have to run addOne and appendBang one by one, passing and getting the states and result manually.
Although I found it can return another State, the code is not improved much:
def myAction(n: Int): State[(String, Int), String] = State {
case (addOneState: String, appendBangState: Int) =>
val (addOneState2, n2) = addOne(n).runState(addOneState)
val (appendBangState2, n3) = appendBang(n2.toString).runState( appendBangState)
((addOneState2, appendBangState2), n3)
}
Since I'm not quite familiar with them, just wondering is there any way to improve it. The best hope is that I can use for comprehension, but not sure if that's possible
Like I mentioned in my first comment, it will be impossible to use a for comprehension to do what you want, because it can not change the type of the state (S).
Remember that a for comprehension can be translated to a combination of flatMaps, withFilter and one map. If we look at your State.flatMap, it takes a function f to change a State[S,A] into State[S, B]. We can use flatMap and map (and thus a for comprehension) to chain together operations on the same state, but we can't change the type of the state in this chain.
We could generalize your last definition of myAction to combine, compose, ... two functions using state of a different type. We can try to implement this generalized compose method directly in our State class (although this is probably so specific, it probably doesn't belong in State). If we look at State.flatMap and myAction we can see some similarities:
We first call runState on our existing State instance.
We then call runState again
In myAction we first use the result n2 to create a State[Int, String] (AppendBang[String] or State[S2, B]) using the second function (appendBang or f) on which we then call runState. But our result n2 is of type String (A) and our function appendBang needs an Int (B) so we need a function to convert A into B.
case class State[S, A](runState: S => (S, A)) {
// flatMap and map
def compose[B, S2](f: B => State[S2, B], convert: A => B) : State[(S, S2), B] =
State( ((s: S, s2: S2) => {
val (sNext, a) = runState(s)
val (s2Next, b) = f(convert(a)).runState(s2)
((sNext, s2Next), b)
}).tupled)
}
You then could define myAction as :
def myAction(i: Int) = addOne(i).compose(appendBang, _.toString)
val twoStates = myAction(1)
// State[(String, Int),String] = State(<function1>)
twoStates.runState(("", 1))
// ((String, Int), String) = ((.,2),2 !!!)
If you don't want this function in your State class you can create it as an external function :
def combineStateFunctions[S1, S2, A, B](
a: A => State[S1, A],
b: B => State[S2, B],
convert: A => B
)(input: A): State[(S1, S2), B] = State(
((s1: S1, s2: S2) => {
val (s1Next, temp) = a(input).runState(s1)
val (s2Next, result) = b(convert(temp)).runState(s2)
((s1Next, s2Next), result)
}).tupled
)
def myAction(i: Int) =
combineStateFunctions(addOne, appendBang, (_: Int).toString)(i)
Edit : Bergi's idea to create two functions to lift a State[A, X] or a State[B, X] into a State[(A, B), X].
object State {
def onFirst[A, B, X](s: State[A, X]): State[(A, B), X] = {
val runState = (a: A, b: B) => {
val (nextA, x) = s.runState(a)
((nextA, b), x)
}
State(runState.tupled)
}
def onSecond[A, B, X](s: State[B, X]): State[(A, B), X] = {
val runState = (a: A, b: B) => {
val (nextB, x) = s.runState(b)
((a, nextB), x)
}
State(runState.tupled)
}
}
This way you can use a for comprehension, since the type of the state stays the same ((A, B)).
def myAction(i: Int) = for {
x <- State.onFirst(addOne(i))
y <- State.onSecond(appendBang(x.toString))
} yield y
myAction(1).runState(("", 1))
// ((String, Int), String) = ((.,2),2 !!!)
This is a followup to my previous question with an example found on the Internet.
Suppose I define a typeclass Applicative as follows:
trait Functor[T[_]]{
def map[A,B](f:A=>B, ta:T[A]):T[B]
}
trait Applicative[T[_]] extends Functor[T] {
def unit[A](a:A):T[A]
def ap[A,B](tf:T[A=>B], ta:T[A]):T[B]
}
I can define an instance of Applicative for List
object AppList extends Applicative[List] {
def map[A,B](f:A=>B, as:List[A]) = as.map(f)
def unit[A](a: A) = List(a)
def ap[A,B](fs:List[A=>B], as:List[A]) = for(f <- fs; a <- as) yield f(a)
}
For convenience I can define an implicit conversion to add a method <*> to List[A=>B]
implicit def toApplicative[A, B](fs: List[A=>B]) = new {
def <*>(as: List[A]) = AppList.ap(fs, as)
}
Now I can do a cool thing !
zip two lists List[String] and apply f2 to every pair in applicative style
val f2: (String, String) => String = {(first, last) => s"$first $last"}
val firsts = List("a", "b", "c")
val lasts = List("x", "y", "z")
scala> AppList.unit(f2.curried) <*> firsts <*> lasts
res31: List[String] = List(a x, a y, a z, b x, b y, b z, c x, c y, c z)
So far, so good but now I have:
val firstsOpt = Some(firsts)
val lastsOpt = Some(lasts)
I would like to zip firsts and lasts, apply f2, and get Option[List[String]] in applicative style. In other words I need <*> for Option[List[_]]. How can I do it ?
Firstly, you need an instance of applicative for Option:
implicit object AppOption extends Applicative[Option] {
def map[A, B](f: A => B, o: Option[A]) = o.map(f)
def unit[A](a: A): Option[A] = Some(a)
def ap[A, B](of: Option[A => B], oa: Option[A]) = of match {
case Some(f) => oa.map(f)
case None => None
}
}
Then you can also create an applicative instance for the composition of two applicatives (note, based on the Haskell version):
class AppComp[F[_], G[_]](fa: Applicative[F], ga: Applicative[G]) extends Applicative[({ type f[A] = F[G[A]]})#f] {
def map[A, B](f: A => B, a: F[G[A]]): F[G[B]] = fa.map((g: G[A]) => ga.map(f, g), a)
def unit[A](a: A) = fa.unit(ga.unit(a))
def ap[A, B](f: F[G[A => B]], a: F[G[A]]): F[G[B]] = {
val liftg: G[A => B] => (G[A] => G[B]) = gf => (gx => ga.ap(gf, gx))
val ffg: F[G[A] => G[B]] = fa.map(liftg, f)
fa.ap(ffg, a)
}
}
implicit def toComp[F[_], G[_]](implicit fa: Applicative[F], ga: Applicative[G]) = new AppComp[F, G](fa, ga)
Finally you can now do:
val ola = toComp[Option, List]
ola.ap(ola.ap(ola.unit(f2.curried), firstsOpt), lastsOpt)
You could probably also remove some of the noise by generalising <*> to work for any applicative.
This is a follow-up to my previous quesion. I am reading this post again to understand the design described there.
They introduce a Job monad similar to Haxl. Job[T] is a data fetch operation that fetches data of type T (and may consist of other operations, i.e. it is a data fetches sequence).
sealed trait Job[+T]
case class PureJob[T](value: T) extends Job[T]
case class BlockedJob[S,T](f: S => Job[T], job: Job[S]) extends Job[T]
case class FetchJob[T](url: Url) extends Job[T]
def pure[T](value: T): Job[T] = PureJob(value)
def flatMap[S,T](f: S => Job[T], job: Job[S]): Job[T] =
job match {
case PureJob(value) => f(value)
case _ => BlockedJob(f, job)
}
They introduce also a function execute to actually execute a Job[T] operation and return a future.
def execute[T](job: Job[T])(implicit ec: ExecutionContext): Future[T] = { ... }
For concurrent data fetching they add new PairJob, and MapJob:
case class MapJob[S, T](f: S => T, job: Job[S]) extends Job[T]
case class PairJob[S, T](jobS: Job[S], jobT: Job[T]) extends Job[(S, T)]
Now they can write:
val jobA: FetchJob[A] = ...
val jobB: FetchJob[B] = ...
val f: A => B = ...
// jobAB is a MapJob of "f" and PairJob of jobA and jobB
val jobAB = (jobA |#| jobB) {(a, b) => f(a, b)}
My question is how to define Job[T] as Applicative to write code as in the example above.
Well has soon has you have PairJob, you have what you need for applicative.
With any generic type G, (here, that would be Job)
if you have pairing:
def pair[A,B](ga: G[A], gb: G[B]): G[(A,B)]
(which for Job, is just PairJob(ga, gb))
and also map, then you can easily implement apply
def ap[A, B](ga: ⇒ G[A])(f: ⇒ G[A ⇒ B]): G[B] = {
val argAndFunc : G[(A, A => B)] = pair(ga, f)
argAndFunc map {case (arg, func) => func(arg)}
}
The reverse is true, if you have ap and map, you easily get pair
def pair[A,B](ga: G[A], gb: G[B]): G[(A,B)] = {
val pairWithB : G[B => (A,B]) = ga map {a => b: B => (a,b)}
ap(gb)(pairWithB)
}
you can always define map in terms of flatMap and pure:
def map[A,B](fa: Job[A])(f: A=>B): Job[B] = fa flatMap (f andThen pure)
and then you can define ap in terms of map and flatMap:
def ap[A,B](fa: => Job[A])(f: => Job[A => B]): Job[B] =
fa flatMap { a =>
f map (_(a))
}
or with for comprehension sugar:
def ap[A,B](fa: => Job[A])(f: => Job[A => B]): Job[B] =
for {
a <- fa
ff <- f
} yield ff(a)
or you can skip map:
def ap[A,B](fa: => Job[A])(f: => Job[A => B]): Job[B] =
fa flatMap { a =>
f flatMap { ff => pure(ff(a)) }
}
How is it possible to create Enumerator for BufferedReader?
I found rather old article: http://apocalisp.wordpress.com/2010/10/17/scalaz-tutorial-enumeration-based-io-with-iteratees/ and it looks like it doesn't work with Scalaz 6.0.4
I try to create Enumerator based on example from here: Idiomatic construction to check whether a collection is ordered
implicit val ListEnumerator = new Enumerator[List] {
def apply[E, A](e: List[E], i: IterV[E, A]): IterV[E, A] = e match {
case List() => i
case x :: xs => i.fold(done = (_, _) => i,
cont = k => apply(xs, k(El(x))))
}
}
But I can't understand how to combine IO monad with Enumerator
What's wrong with Rúnar's article? The following version is working for me (Scalaz 6.0.4):
object FileIteratee {
def enumReader[A](r: BufferedReader, it: IterV[String, A]) : IO[IterV[String, A]] = {
def loop: IterV[String, A] => IO[IterV[String, A]] = {
case i#Done(_, _) => i.pure[IO]
case i#Cont(k) => for {
s <- r.readLine.pure[IO]
a <- if (s == null) i.pure[IO] else loop(k(El(s)))
} yield a
}
loop(it)
}
def bufferFile(f: File) = new BufferedReader(new FileReader(f)).pure[IO]
def closeReader(r: Reader) = r.close().pure[IO]
def bracket[A,B,C](init: IO[A], fin: A => IO[B], body: A => IO[C]): IO[C] =
for {
a <- init
c <- body(a)
_ <- fin(a)
} yield c
def enumFile[A](f: File, i: IterV[String, A]) : IO[IterV[String, A]] =
bracket(bufferFile(f),
closeReader(_: BufferedReader),
enumReader(_: BufferedReader, i))
}