How to compose two different `State Monad`? - scala

When I learn State Monad, I'm not sure how to compose two functions with different State return types.
State Monad definition:
case class State[S, A](runState: S => (S, A)) {
def flatMap[B](f: A => State[S, B]): State[S, B] = {
State(s => {
val (s1, a) = runState(s)
val (s2, b) = f(a).runState(s1)
(s2, b)
})
}
def map[B](f: A => B): State[S, B] = {
flatMap(a => {
State(s => (s, f(a)))
})
}
}
Two different State types:
type AppendBang[A] = State[Int, A]
type AddOne[A] = State[String, A]
Two methods with differnt State return types:
def addOne(n: Int): AddOne[Int] = State(s => (s + ".", n + 1))
def appendBang(str: String): AppendBang[String] = State(s => (s + 1, str + " !!!"))
Define a function to use the two functions above:
def myAction(n: Int) = for {
a <- addOne(n)
b <- appendBang(a.toString)
} yield (a, b)
And I hope to use it like this:
println(myAction(1))
The problem is myAction is not compilable, it reports some error like this:
Error:(14, 7) type mismatch;
found : state_monad.State[Int,(Int, String)]
required: state_monad.State[String,?]
b <- appendBang(a.toString)
^
How can I fix it? Do I have to define some Monad transformers?
Update: The question may be not clear, let me give an example
Say I want to define another function, which uses addOne and appendBang internally. Since they all need existing states, I have to pass some to it:
def myAction(n: Int)(addOneState: String, appendBangState: Int): ((String, Int), String) = {
val (addOneState2, n2) = addOne(n).runState(addOneState)
val (appendBangState2, n3) = appendBang(n2.toString).runState(appendBangState)
((addOneState2, appendBangState2), n3)
}
I have to run addOne and appendBang one by one, passing and getting the states and result manually.
Although I found it can return another State, the code is not improved much:
def myAction(n: Int): State[(String, Int), String] = State {
case (addOneState: String, appendBangState: Int) =>
val (addOneState2, n2) = addOne(n).runState(addOneState)
val (appendBangState2, n3) = appendBang(n2.toString).runState( appendBangState)
((addOneState2, appendBangState2), n3)
}
Since I'm not quite familiar with them, just wondering is there any way to improve it. The best hope is that I can use for comprehension, but not sure if that's possible

Like I mentioned in my first comment, it will be impossible to use a for comprehension to do what you want, because it can not change the type of the state (S).
Remember that a for comprehension can be translated to a combination of flatMaps, withFilter and one map. If we look at your State.flatMap, it takes a function f to change a State[S,A] into State[S, B]. We can use flatMap and map (and thus a for comprehension) to chain together operations on the same state, but we can't change the type of the state in this chain.
We could generalize your last definition of myAction to combine, compose, ... two functions using state of a different type. We can try to implement this generalized compose method directly in our State class (although this is probably so specific, it probably doesn't belong in State). If we look at State.flatMap and myAction we can see some similarities:
We first call runState on our existing State instance.
We then call runState again
In myAction we first use the result n2 to create a State[Int, String] (AppendBang[String] or State[S2, B]) using the second function (appendBang or f) on which we then call runState. But our result n2 is of type String (A) and our function appendBang needs an Int (B) so we need a function to convert A into B.
case class State[S, A](runState: S => (S, A)) {
// flatMap and map
def compose[B, S2](f: B => State[S2, B], convert: A => B) : State[(S, S2), B] =
State( ((s: S, s2: S2) => {
val (sNext, a) = runState(s)
val (s2Next, b) = f(convert(a)).runState(s2)
((sNext, s2Next), b)
}).tupled)
}
You then could define myAction as :
def myAction(i: Int) = addOne(i).compose(appendBang, _.toString)
val twoStates = myAction(1)
// State[(String, Int),String] = State(<function1>)
twoStates.runState(("", 1))
// ((String, Int), String) = ((.,2),2 !!!)
If you don't want this function in your State class you can create it as an external function :
def combineStateFunctions[S1, S2, A, B](
a: A => State[S1, A],
b: B => State[S2, B],
convert: A => B
)(input: A): State[(S1, S2), B] = State(
((s1: S1, s2: S2) => {
val (s1Next, temp) = a(input).runState(s1)
val (s2Next, result) = b(convert(temp)).runState(s2)
((s1Next, s2Next), result)
}).tupled
)
def myAction(i: Int) =
combineStateFunctions(addOne, appendBang, (_: Int).toString)(i)
Edit : Bergi's idea to create two functions to lift a State[A, X] or a State[B, X] into a State[(A, B), X].
object State {
def onFirst[A, B, X](s: State[A, X]): State[(A, B), X] = {
val runState = (a: A, b: B) => {
val (nextA, x) = s.runState(a)
((nextA, b), x)
}
State(runState.tupled)
}
def onSecond[A, B, X](s: State[B, X]): State[(A, B), X] = {
val runState = (a: A, b: B) => {
val (nextB, x) = s.runState(b)
((a, nextB), x)
}
State(runState.tupled)
}
}
This way you can use a for comprehension, since the type of the state stays the same ((A, B)).
def myAction(i: Int) = for {
x <- State.onFirst(addOne(i))
y <- State.onSecond(appendBang(x.toString))
} yield y
myAction(1).runState(("", 1))
// ((String, Int), String) = ((.,2),2 !!!)

Related

Programming a state monad in Scala

The theory of how a state monad looks like I borrow from Philip Wadler's Monads for Functional Programming:
type M a = State → (a, State)
type State = Int
unit :: a → M a
unit a = λx. (a, x)
(*) :: M a → (a → M b) → M b
m * k = λx.
let (a, y) = m x in
let (b, z) = k a y in
(b, z)
The way I would like to use a state monad is as follows:
Given a list L I want different parts of my code to get this list and update this list by adding new elements at its end.
I guess the above would be modified as:
type M = State → (List[Data], State)
type State = List[Data]
def unit(a: List[Data]) = (x: State) => (a,x)
def star(m: M, k: List[Data] => M): M = {
(x: M) =>
val (a,y) = m(x)
val (b,z) = k(a)(y)
(b,z)
}
def get = ???
def update = ???
How do I fill in the details, i.e.?
How do I instantiate my hierarchy to work on a concrete list?
How do I implement get and update in terms of the above?
Finally, how would I do this using Scala's syntax with flatMap and unit?
Your M is defined incorrectly. It should take a/A as a parameter, like so:
type M[A] = State => (A, State)
You've also missed that type parameter elsewhere.
unit should have a signature like this:
def unit[A](a: A): M[A]
star should have a signature like this:
def star[A, B](m: M[A], k: A => M[B]): M[B]
Hopefully, that makes the functions more clear.
Your implementation of unit was pretty much the same:
def unit[A](a: A): M[A] = x => (a, x)
However, in star, the parameter of your lambda (x) is of type State, not M, because M[B] is basically State => (A, State). The rest you got right:
def star[A, B](m: M[A])(k: A => M[B]): M[B] =
(x: State) => {
val (a, y) = m(x)
val (b, z) = k(a)(y)
(b, z)
}
Edit: According to #Luis Miguel Mejia Suarez:
It would probably be easier to implement if you make your State a class and define flatMap inside it. And you can define unit in the companion object.
He suggested final class State[S, A](val run: S => (A, S)), which would also allow you to use infix functions like >>=.
Another way to do it would be to define State as a type alias for a function S => (A, S) and extend it using an implicit class.
type State[S, A] = S => (A, S)
object State {
//This is basically "return"
def unit[S, A](a: A): State[S, A] = s => (a, s)
}
implicit class StateOps[S, A](private runState: S => (A, S)) {
//You can rename this to ">>=" or "flatMap"
def *[B](k: A => State[S, B]): State[S, B] = s => {
val (a, s2) = runState(s)
k(a)(s2)
}
}
If your definition of get is
set the result value to the state and leave the state unchanged
(borrowed from Haskell Wiki), then you can implement it like this:
def get[S]: State[S, S] = s => (s, s)
If you mean that you want to extract the state (in this case a List[Data]), you can use execState (define it in StateOps):
def execState(s: S): S = runState(s)._2
Here's a terrible example of how you can add elements to a List.
def addToList(n: Int)(list: List[Int]): ((), List[Int]) = ((), n :: list)
def fillList(n: Int): State[List[Int], ()] =
n match {
case 0 => s => ((), s)
case n => fillList(n - 1) * (_ => addToList(n))
}
println(fillList(10)(List.empty)) gives us this (the second element can be extracted with execState):
((),List(10, 9, 8, 7, 6, 5, 4, 3, 2, 1))

Is it possible to combine multiple map and reduce functions into a single pass in Scala?

I have multiple map functions running over the same data and I'd like to have them run in a single pass. I'm looking for a generic way to do this.
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if(c1 > c2) c1 else c2
def mapG(s: String): Int = s.length
def reduceG(i1: Int, i2: Int): Int = i1 + i2
val largestStartingChar = fruits.map(mapF).reduce(reduceF)
val totalStringLength = fruits.map(mapG).reduce(reduceG)
I'd like to reduce the number of passes over fruits. I can make this generic for two maps and reduces like this:
def productMapFunction[A, B, C](f: A=>B, g: A=>C): A => (B, C) = {
x => (f(x), g(x))
}
def productReduceFunction[T, U](f: (T, T)=>T, g: (U, U) => U):
((T,U), (T,U)) => (T, U) = {
(tu1, tu2) => (f(tu1._1, tu2._1), g(tu1._2, tu2._2))
}
val xMapFG = productMapFunction(mapF, mapG)
val xReduceFG = productReduceFunction(reduceF, reduceG)
val (largestStartingChar2, totalStringLength2) =
fruits.map(xMapFG).reduce(xReduceFG))
I'd like to do this even more generically, with an arbitrary number of map and reduce functions, but I'm not sure how to proceed, or if this is possible.
Following solution uses Cats 2 and custom type MapReduce.
Reducing operation can be specified by function reduce: (O, O) => O
or cats reducer: Semigroup[O].
Multiple MapReduce objects can be combined into one by Apply instance provided by implicit def mapReduceApply[I]
import cats._
import cats.implicits._
trait MapReduce[I, O] {
type R
def reducer: Semigroup[R]
def map: I => R
def mapResult: R => O
def apply(input: Seq[I]): O = mapResult(input.map(map).reduce(reducer.combine))
}
object MapReduce {
def apply[I, O, _R](_reducer: Semigroup[_R], _map: I => _R, _mapResult: _R => O): MapReduce[I, O] =
new MapReduce[I, O] {
override type R = _R
override def reducer = _reducer
override def map = _map
override def mapResult = _mapResult
}
def apply[I, O](map: I => O)(implicit r: Semigroup[O]): MapReduce[I, O] =
MapReduce[I, O, O](r, map, identity)
def apply[I, O](map: I => O, reduce: (O, O) => O): MapReduce[I, O] = {
val reducer = new Semigroup[O] {
override def combine(x: O, y: O): O = reduce(x, y)
}
MapReduce(map)(reducer)
}
implicit def mapReduceApply[I] =
new Apply[({type F[X] = MapReduce[I, X]})#F] {
override def map[A, B](f: MapReduce[I, A])(fn: A => B): MapReduce[I, B] =
MapReduce(f.reducer, f.map, f.mapResult.andThen(fn))
override def ap[A, B](ff: MapReduce[I, (A) => B])(fa: MapReduce[I, A]): MapReduce[I, B] =
MapReduce(ff.reducer product fa.reducer,
i => (ff.map(i), fa.map(i)),
(t: (ff.R, fa.R)) => ff.mapResult(t._1)(fa.mapResult(t._2))
)
}
}
object MultiMapReduce extends App {
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if (c1 > c2) c1 else c2
val biggestFirsChar = MapReduce(mapF, reduceF)
val totalChars = MapReduce[String, Int](_.length) // (Semigroup[Int]) reduce by _ + _
def count[A] = MapReduce[A, Int](_ => 1)
val multiMapReduce = (biggestFirsChar, totalChars, count[String]).mapN((_, _, _))
println(multiMapReduce(fruits))
val sum = MapReduce[Double, Double](identity)
val average = (sum, count[Double]).mapN(_ / _)
println(sum(List(1, 2, 3, 4)))
println(average(List(1, 2, 3, 4)))
}
Runnable version is also available on GitHub.
Interesting question!
I don't know of any such implementation in the standard library or even scalaz/cats.
It's not very surprising because if your list is not very large you can just perform map-reduces sequentially and I'm not even sure that overhead of constructing lots of intermediate objects would be smaller than overhead of traversing the list several times.
And if the list is potentially doesn't fit into memory you should be using one of the streaming libraries (fs2/zio-streams/akka-streams)
Although if your input was Iterator instead of List, such functionality would be useful.
There is an interesting article about this problem:
https://softwaremill.com/beautiful-folds-in-scala/
tldr:
Map-reduce workflow could be formalized as follows:
trait Fold[I, O] {
type M
def m: Monoid[M]
def tally: I => M
def summarize: M => O
}
In your case I = List[A], tally = list => list.map(mapF), summarize = list => list.reduce(reduceF) .
To run a map-reduce on a list using an instance of fold you need to run
fold.summarize(fold.tally(list))
You can define combine operation on them:
def combine[I, O1, O2](f1: Fold[I, O1], f2: Fold[I, O2]): Fold[I, (O1, O2)]
Using combine few times would give you what you want:
combine(combine(f1, f2), f3): Fold[I, ((O1, O2), O3)]
I think you're just trying to reinvent transducers. It's been a while since I have used Scala, but there's at least one implementation.

is it possible to implement flip as a Scala function (and not a method)

As a part of learning Scala I try to implement Haskell's flip function (a function with signature (A => B => C) => (B => A => C)) in Scala - and implement it as a function (using val) and not as a method (using def).
I can implement it as a method, for instance this way:
def flip[A, B, C](f: (A, B) => C):((B, A) => C) = (b: B, a: A) => f(a, b)
val minus = (a: Int, b: Int) => a - b
val f = flip(minus)
println(f(3, 5))
However, when I try to implement it as a function, it does not work:
val flip = (f: ((Any, Any) => Any)) => ((a: Any, b: Any) => f(b, a))
val minus = (a: Int, b: Int) => a - b
val f = flip(minus)
println(f(3, 5))
When I try to compile this code, it fails with this message:
Error:(8, 18) type mismatch;
found : (Int, Int) => Int
required: (Any, Any) => Any
val f = flip(minus)
I understand why it fails: I try to pass (Int, Int) => Int where (Any, Any) => Any is expected. However, I don't know how to fix this problem. Is it possible at all?
Scala doesn't support polymorphic functions, unlike methods which are. This is due to the first class value nature of functions, which are simply instances of the FunctioN traits. These functions are classes, and they need the types to be bound at declaration site.
If we took the flip method and tried to eta expand it to a function, we'd see:
val flipFn = flip _
We'd get back in return a value of type:
((Nothing, Nothing) => Nothing) => (Nothing, Nothing) => Nothing
Due to the fact that none of the types were bound, hence the compiler resorts to the buttom type Nothing.
However, not all hope is lost. There is a library called shapeless which does allow us to define polymorphic functions via PolyN.
We can implement flip like this:
import shapeless.Poly1
object flip extends Poly1 {
implicit def genericCase[A, B, C] = at[(A, B) => C](f => (b: B, a: A) => f(a, b))
}
flip is no different from the FunctionN trait, it defines an apply method which will be called.
We use it like this:
def main(args: Array[String]): Unit = {
val minus = (a: Int, b: Int) => a - b
val f = flip(minus)
println(f(3, 5))
}
Yielding:
2
This would also work for String:
def main(args: Array[String]): Unit = {
val stringConcat = (a: String, b: String) => a + b
val f = flip(stringConcat)
println(f("hello", "world"))
}
Yielding:
worldhello

compiler error message when using State monad for memoization

I have a problem to make a working version of the Euler project problem 31 with the use of State trait (inspired from scalaz)
First, I have a solution with a mutable HashMap for memoization. It works but i would like to use the State monad, to understand it and to improve my skills.
I have used it with the fibonacci example, but when i attempt to apply the same technique to my case, i have a compiler error that i don't understand.
I use this implementation for State :
trait State[S, A] {
val run: S => (S, A)
def apply(s: S): (S, A) = run(s)
def eval(s: S): A = run(s)._2
def map[B](f: A => B): State[S, B] =
State { s: S =>
val (s1, a) = run(s)
(s1, f(a))
}
def flatMap[B](f: A => State[S, B]): State[S, B] =
State { s: S =>
val (s1, a) = run(s)
f(a)(s1)
}
}
object State {
def apply[S, A](f: S => (S, A)): State[S, A] = new State[S, A] {
final val run = f
}
def init[S, A](a: A) = State { s: S => (s, a) }
def update[S, A](f: S => S): State[S, Unit] = State { s: S => (f(s), ()) }
def gets[S, A](f: S => A): State[S, A] = State { s: S => (s, f(s)) }
}
my attempt to use it is here :
val coins = List(1, 2, 5, 10, 20, 50, 100, 200)
type MemoKey = (List[Int], Int)
type MemoType = Map[MemoKey, Int]
def ways(listCoins: List[Int], amount: Int): Int = {
def ways_impl(coins: List[Int], sum: Int): State[MemoType, Int] = (coins, sum) match {
case (Nil, 0) => State.init(1)
case (Nil, _) => State.init(0)
case (c :: cs, _) =>
for {
memoed <- State.gets { m: MemoType => m.get((coins, sum)) }
res <- memoed match {
case Some(way) => State.init[MemoType, Int](way)
case None =>
(for {
i <- 0 to sum / c
r <- ways_impl(cs, sum - i * c)
_ <- State.update { m: MemoType => m + ((coins, sum) -> r) }
} yield r).sum
}
} yield res
}
ways_impl(listCoins, amount) eval (Map())
I have a compiler error at this line :
r <- ways_impl(cs, sum - i * c)
The compiler said :
type mismatch; found : State[MemoType,Int] (which expands to) State[scala.collection.immutable.Map[(List[Int], Int),Int],Int] required: scala.collection.GenTraversableOnce[?]
For information, here is my first version with mutable map :
import scala.collection.mutable._
val memo = HashMap[(List[Int], Int), Int]()
val coins = List(1, 2, 5, 10, 20, 50, 100, 200)
def memoWays(coins: List[Int], sum: Int): Int = {
memo.getOrElse((coins, sum), {
val y = ways(coins, sum)
memo += ((coins, sum) -> y)
y
})
}
// brute force method with memoization
def ways(coins: List[Int], sum: Int): Int = (coins, sum) match {
case (Nil, 0) => 1
case (Nil, _) => 0
case (c :: cs, n) =>
(for {
i <- 0 to n / c
r = memoWays(cs, n - i * c)
} yield r).sum
}
println(s"result=${Mesure(ways(coins, 200))}")
What does that error mean ? Why the compiler want a GenTraversableOnce instead of State ?
What kind of thing i don't understand on State monad ?
And, if i may, I have an optional question :
Is my way to memoize with State Monad, is a good choice, or my first implementation with mutable map is better anyway ?
The problem is that your for comprehension is attempting to flatMap two unrelated types: a Range and a State. You're going to have to refactor, although off the top of my head, it's not clear to me how you'll be able to leverage State in a simple way. I'd probably use an immutable Map for the memo, a List to represent the future iterations to be tried, and simple recursion to iterate.

Using lambda with generic type in Scala

The second argument of myFunc is a function with complex arguments:
def myFunc(list : List[String],
combine: (Map[String, ListBuffer[String]], String, String) => Unit) = {
// body of myFunc is just a stub and doesn't matter
val x = Map[String, ListBuffer[String]]()
list.foreach ((e:String) => {
val spl = e.split(" ")
combine(x, spl(0), spl(1))
})
x
}
I need to pass second argument to myFunc, so it can be used with various types A, B instead of specific String, ListBuffer[String].
def myFunc(list : List[A], combine: (Map[A, B], A, A) => Unit) = {
val x = Map[A, B]()
list.foreach(e => {
combine(x, e)
})
}
How to declare and call such construct?
You can do the following,
def myFunc[A, B](list : List[A], combine: (Map[A, B], A, A) => Unit) = {
val x = Map[A, B]()
list.foreach (e => combine(x, e, e))
x
}
Ad use it like
myFunc[String, Int](List("1","2","3"), (obj, k, v) => obj.put(k, v.toInt) )
It seems that you are looking to generalise the container being used. Were you looking for something like this? Here we import scala.language.higherKinds so that we can take Container, a kind which takes a single type parameter as a type parameter to addPair.
import scala.language.higherKinds
def addPair[K, V, Container[_]](map: Map[K, Container[V]],
addToContainer: (Container[V], V) => Container[V],
emptyContainer: => Container[V],
pair: (K, V)): Map[K, Container[V]] = {
val (key, value) = pair
val existingValues = map.getOrElse(key, emptyContainer)
val newValues = addToContainer(existingValues, value)
map + (key -> newValues)
}