Related
I'm trying to write a DSL for writing system tests in Scala. In this DSL I don't want to expose the fact that some operations might take place asynchronously (because they are implemented using the web-service under test for instance), or that errors might occur (because the web-service might not be available, and we want the test to fail). In this answer this approach is discouraged, but I don't completely agree with this in the context of a DSL for writing tests. I think the DSL will get unnecessary polluted by the introduction of these aspects.
To frame the question, consider the following DSL:
type Elem = String
sealed trait TestF[A]
// Put an element into the bag.
case class Put[A](e: Elem, next: A) extends TestF[A]
// Count the number of elements equal to "e" in the bag.
case class Count[A](e: Elem, withCount: Int => A) extends TestF[A]
def put(e: Elem): Free[TestF, Unit] =
Free.liftF(Put(e, ()))
def count(e: Elem): Free[TestF, Int] =
Free.liftF(Count(e, identity))
def test0 = for {
_ <- put("Apple")
_ <- put("Orange")
_ <- put("Pinneaple")
nApples <- count("Apple")
nPears <- count("Pear")
nBananas <- count("Banana")
} yield List(("Apple", nApples), ("Pears", nPears), ("Bananas", nBananas))
Now assume we want to implement an interpreter that makes use of our service under test to put and count the elements in the store. Since we make use of the network, I'd like that the put operations take place asynchronously. In addition, given that network errors or server errors can occur, I'd like the program to stop as soon as an error occurs. To give an idea of what I want to achieve, here is an example of mixing the different aspects in Haskell by means of monad transformers (that I cannot translate to Scala).
So my question is, which monad M would you use for an interpreter that satisfies the requirements above:
def interp[A](cmd: TestF[A]): M[A]
And in case M is a monad tranformer, how would you compose them using the FP library of your choice (Cats, Scalaz).
Task (scalaz or better fs2) should satisfy all of the requirements, it doesn't need monad-transformer as it's already has Either inside (Either for fs2, \/ for scalaz). It also has a fast-fail behavior you need, same as right-biased disjunction/xor.
Here are several implementations that are known to me:
Scalaz Task (original): little outdated doc and new sources
FS2 Task: https://github.com/functional-streams-for-scala/fs2/blob/series/0.9/docs/guide.md It also provides interoperability (type classes) with scalaz and cats
Monix Task: https://monix.io/docs/2x/eval/task.html
"Cats" doesn't provide any Task or other IO-monad-related operations (no scalaz-effect analog at all) and recommends to use either Monix or FS2.
Regardless of monad-transformer absence, you still kinda need lifting when using Task:
from value to Task or
from Either to Task
But yes, it does seem to be simpler than monad transformers especially in respect to the fact monads are hardly composable - in order to define monad transformer you have to know some other details about your type besides being a monad (usually it requires something like comonad to extract value).
Just for advertising purposes, I would also add that Task represents stack-safe trampolined computation.
However, there are some projects focused on extended monadic composition, like Emm-monad: https://github.com/djspiewak/emm, so you can compose monad transformers with Future/Task, Either, Option, List and so on and so forth. But, IMO, it's still limited in comparison with Applicative composition - cats provides universal Nested data type that allows to easily compose any Applicative, you can find some examples in this answer - the only disadvantage here is that it's hard to build a readable DSL using Applicative. Another alternative is so-called "Freer monad": https://github.com/m50d/paperdoll, which basically provides better composition and allows to separate different effect layers into different interpreters.
For example, as there is no FutureT/TaskT transformer you can't build effects like type E = Option |: Task |: Base (Option from Task) as such flatMap would require extraction of value from the Future/Task.
As a conclusion, I can say that from my experience Task really comes in hand for do-notation based DSLs: I had a complex external rule-like DSL for async computations and when I decided to migrate it all to Scala-embedded version Task really helped - I literally converted external-DSL to Scala's for-comprehension. Another thing we considered is having some custom type, like ComputationRule with a set of type classes defined over it along with conversions to Task/Future or whatever we need, but this was because we didn't use Free-monad explicitly.
You might even not need Free-monad here assuming you don't need an ability to switch interpreters (which might be true for just system tests). In that case Task might be the only thing you need - it's lazy (in comparison with Future), truly functional and stack-safe:
trait DSL {
def put[E](e: E): Task[Unit]
def count[E](e: E): Task[Int]
}
object Implementation1 extends DSL {
...implementation
}
object Implementation2 extends DSL {
...implementation
}
//System-test script:
def test0(dsl: DSL) = {
import dsl._
for {
_ <- put("Apple")
_ <- put("Orange")
_ <- put("Pinneaple")
nApples <- count("Apple")
nPears <- count("Pear")
nBananas <- count("Banana")
} yield List(("Apple", nApples), ("Pears", nPears), ("Bananas", nBananas))
}
So you can switch implementation by passing different "interpreter" here:
test0(Implementation1).unsafeRun
test0(Implementation2).unsafeRun
Differences/Disadvantages (in comparison with http://typelevel.org/cats/datatypes/freemonad.html):
you stuck with Task type, so you can't collapse it to some other monad easily.
implementation is resolved in runtime when you pass an instance of DSL-trait (instead of natural transformation), you can easily abstract it using eta-expansion: test0 _. Polymorphic methods (put, count) are naturally supported by Java/Scala, but poly functions aren't so it's easier to pass instance of DSL containing T => Task[Unit] (for put operation) than making synthetic polymorphic function DSLEntry[T] => Task[Unit] using natural-transform DSLEntry ~> Task.
no explicit AST as instead of pattern matching inside natural transformation - we use static dispatch (explicitly calling a method, which will return lazy computation) inside DSL trait
Actually, you can even get rid of Task here:
trait DSL[F[_]] {
def put[E](e: E): F[Unit]
def count[E](e: E): F[Int]
}
def test0[M[_]: Monad](dsl: DSL[M]) = {...}
So here it might even become a matter of preference especially when you're not writing an open-source library.
Putting it all together:
import cats._
import cats.implicits._
trait DSL[F[_]] {
def put[E](e: E): F[Unit]
def count[E](e: E): F[Int]
}
def test0[M[_]: Monad](dsl: DSL[M]) = {
import dsl._
for {
_ <- put("Apple")
_ <- put("Orange")
_ <- put("Pinneaple")
nApples <- count("Apple")
nPears <- count("Pear")
nBananas <- count("Banana")
} yield List(("Apple", nApples), ("Pears", nPears), ("Bananas", nBananas))
}
object IdDsl extends DSL[Id] {
def put[E](e: E) = ()
def count[E](e: E) = 5
}
Note that cats have a Monad defined for Id, so:
scala> test0(IdDsl)
res2: cats.Id[List[(String, Int)]] = List((Apple,5), (Pears,5), (Bananas,5))
simply works. Of course, you can choose Task/Future/Option or any combination if you prefer. As a matter of fact, you can use Applicative instead of Monad:
def test0[F[_]: Applicative](dsl: DSL[F]) =
dsl.count("Apple") |#| dsl.count("Pinapple apple pen") map {_ + _ }
scala> test0(IdDsl)
res8: cats.Id[Int] = 10
|#| is a parallel operator, so you can use cats.Validated instead of Xor, be aware that |#| for Task isn't executed (at least in older scalaz version) in parallel (parallel operator not equals parallel computation). You can also use a combination of both:
import cats.syntax._
def test0[M[_]:Monad](d: DSL[M]) = {
for {
_ <- d.put("Apple")
_ <- d.put("Orange")
_ <- d.put("Pinneaple")
sum <- d.count("Apple") |#| d.count("Pear") |#| d.count("Banana") map {_ + _ + _}
} yield sum
}
scala> test0(IdDsl)
res18: cats.Id[Int] = 15
Lets say I have the following free monad definition:
sealed trait SetOpsA[A]
case class AllSetNames() extends SetOpsA[Set[String]]
case class IdsFromSet(setName: String) extends SetOpsA[Set[String]]
object SetOps {
type SetOps[A] = Free[SetOpsA, A]
def allSetNames(): SetOps[Set[String]] =
liftF[SetOpsA, Set[String]](AllSetNames())
def idsFromSet(setName: String): SetOps[Set[String]] =
liftF[SetOpsA, Set[String]](IdsFromSet(setName))
}
Now I would like to retrieve all set names and then for each of these set names execute the idsFromSet action.
I would describe it like this:
val prog = for {
allSetNames <- allSetNames()
setName <- allSetNames
ids <- idsFromSet(setName)
} yield ids
But this is not possible since monads can't be mixed in for comprehensions. So on the line 'setName <- allSetNames' the compiler complains that it is given a Set and it wants a Free.
By looking around it seems that a monad transformer would be the way to go. But with this information I am really lost since the examples with monad transformers I can find seem to be too far away from my example or the explanations too abstract for me to implment it in this example. So some help with this would be appreciated.
Edit: I am a step further and can sort of achieve my result by doing this (I also realized I want the flattened ids, and not a collection of collection of ids):
val prog = for {
allSetNames <- allSetNames()
ids <- allSetNames.toList.traverseU(idsFromSet)
} yield ids.flatten
Why and how specifically is a Scala Future not a Monad; and would someone please compare it to something that is a Monad, like an Option?
The reason I'm asking is Daniel Westheide's The Neophyte's Guide to Scala Part 8: Welcome to the Future where I asked whether or not a Scala Future was a Monad, and the author responded that it wasn't, which threw off base. I came here to ask for a clarification.
A summary first
Futures can be considered monads if you never construct them with effectful blocks (pure, in-memory computation), or if any effects generated are not considered as part of semantic equivalence (like logging messages). However, this isn't how most people use them in practice. For most people using effectful Futures (which includes most uses of Akka and various web frameworks), they simply aren't monads.
Fortunately, a library called Scalaz provides an abstraction called Task that doesn't have any problems with or without effects.
A monad definition
Let's review briefly what a monad is. A monad must be able to define at least these two functions:
def unit[A](block: => A)
: Future[A]
def bind[A, B](fa: Future[A])(f: A => Future[B])
: Future[B]
And these functions must statisfy three laws:
Left identity: bind(unit(a))(f) ≡ f(a)
Right identity: bind(m) { unit(_) } ≡ m
Associativity: bind(bind(m)(f))(g) ≡ bind(m) { x => bind(f(x))(g) }
These laws must hold for all possible values by definition of a monad. If they don't, then we simply don't have a monad.
There are other ways to define a monad that are more or less the same. This one is popular.
Effects lead to non-values
Almost every usage of Future that I've seen uses it for asychronous effects, input/output with an external system like a web service or a database. When we do this, a Future isn't even a value, and mathematical terms like monads only describe values.
This problem arises because Futures execute immediately upon data contruction. This messes up the ability to substitute expressions with their evaluated values (which some people call "referential transparency"). This is one way to understand why Scala's Futures are inadequate for functional programming with effects.
Here's an illustration of the problem. If we have two effects:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits._
def twoEffects =
( Future { println("hello") },
Future { println("hello") } )
we will have two printings of "hello" upon calling twoEffects:
scala> twoEffects
hello
hello
scala> twoEffects
hello
hello
But if Futures were values, we should be able to factor out the common expression:
lazy val anEffect = Future { println("hello") }
def twoEffects = (anEffect, anEffect)
But this doesn't give us the same effect:
scala> twoEffects
hello
scala> twoEffects
The first call to twoEffects runs the effect and caches the result, so the effect isn't run the second time we call twoEffects.
With Futures, we end up having to think about the evaluation policy of the language. For instance, in the example above, the fact I use a lazy value rather than a strict one makes a difference in the operational semantics. This is exactly the kind of twisted reasoning functional programming is designed to avoid -- and it does it by programming with values.
Without substitution, laws break
In the presense of effects, monad laws break. Superficially, the laws appear to hold for simple cases, but the moment we begin to substitute expressions with their evaluated values, we end up with the same problems we illustrated above. We simply can't talk about mathematical concepts like monads when we don't have values in the first place.
To put it bluntly, if you use effects with your Futures, saying they're monads is not even wrong because they aren't even values.
To see how monad laws break, just factor out your effectful Future:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits._
def unit[A]
(block: => A)
: Future[A] =
Future(block)
def bind[A, B]
(fa: Future[A])
(f: A => Future[B])
: Future[B] =
fa flatMap f
lazy val effect = Future { println("hello") }
Again, it will only run one time, but you need it to run twice -- once for the right-hand side of the law, and another for the left. I'll illustrate the problem for the right identity law:
scala> effect // RHS has effect
hello
scala> bind(effect) { unit(_) } // LHS doesn't
The implicit ExecutionContext
Without putting an ExecutionContext in implicit scope, we can't define either unit or bind in our monad. This is because the Scala API for Futures has these signature:
object Future {
// what we need to define unit
def apply[T]
(body: ⇒ T)
(implicit executor: ExecutionContext)
: Future[T]
}
trait Future {
// what we need to define bind
flatMap[S]
(f: T ⇒ Future[S])
(implicit executor: ExecutionContext)
: Future[S]
}
As a "convenience" to the user, the standard library encourages users to define an execution context in implicit scope, but I think this is a huge hole in the API that just leads to defects. One scope of the computation may have one execution context defined while another scope can have another context defined.
Perhaps you can ignore the problem if you define an instance of unit and bind that pins both operations to a single context and use this instance consistently. But this is not what people do most of the time. Most of the time, people use Futures with for-yield comprehensions that become map and flatMap calls. To make for-yield comprehensions work, an execution context must be defined at some non-global implicit scope (because for-yield doesn't provide a way to specify additional parameters to the map and flatMap calls).
To be clear, Scala lets you use lots of things with for-yield comprehensions that aren't actually monads, so don't believe that you have a monad just because it works with for-yield syntax.
A better way
There's a nice library for Scala called Scalaz that has an abstraction called scalaz.concurrent.Task. This abstraction doesn't run effects upon data construction as the standard library Future does. Furthermore, Task actually is a monad. We compose Task monadically (we can use for-yield comprehensions if we like), and no effects run while we're composing. We have our final program when we have composed a single expression evaluating to Task[Unit]. This ends up being our equivalent of a "main" function, and we can finally run it.
Here's an example illustrating how we can substitute Task expressions with their respective evaluated values:
import scalaz.concurrent.Task
import scalaz.IList
import scalaz.syntax.traverse._
def twoEffects =
IList(
Task delay { println("hello") },
Task delay { println("hello") }).sequence_
We will have two printings of "hello" upon calling twoEffects:
scala> twoEffects.run
hello
hello
And if we factor out the common effect,
lazy val anEffect = Task delay { println("hello") }
def twoEffects =
IList(anEffect, anEffect).sequence_
we get what we'd expect:
scala> twoEffects.run
hello
hello
In fact, it doesn't really matter that whether we use a lazy value or a strict value with Task; we get hello printed out twice either way.
If you want to functionally program, consider using Task everywhere you may use Futures. If an API forces Futures upon you, you can convert the Future to a Task:
import concurrent.
{ ExecutionContext, Future, Promise }
import util.Try
import scalaz.\/
import scalaz.concurrent.Task
def fromScalaDeferred[A]
(future: => Future[A])
(ec: ExecutionContext)
: Task[A] =
Task
.delay { unsafeFromScala(future)(ec) }
.flatMap(identity)
def unsafeToScala[A]
(task: Task[A])
: Future[A] = {
val p = Promise[A]
task.runAsync { res =>
res.fold(p failure _, p success _)
}
p.future
}
private def unsafeFromScala[A]
(future: Future[A])
(ec: ExecutionContext)
: Task[A] =
Task.async(
handlerConversion
.andThen { future.onComplete(_)(ec) })
private def handlerConversion[A]
: ((Throwable \/ A) => Unit)
=> Try[A]
=> Unit =
callback =>
{ t: Try[A] => \/ fromTryCatch t.get }
.andThen(callback)
The "unsafe" functions run the Task, exposing any internal effects as side-effects. So try not to call any of these "unsafe" functions until you've composed one giant Task for your entire program.
I believe a Future is a Monad, with the following definitions:
def unit[A](x: A): Future[A] = Future.successful(x)
def bind[A, B](m: Future[A])(fun: A => Future[B]): Future[B] = fut.flatMap(fun)
Considering the three laws:
Left identity:
Future.successful(a).flatMap(f) is equivalent to f(a). Check.
Right identity:
m.flatMap(Future.successful _) is equivalent to m (minus some possible performance implications). Check.
Associativity
m.flatMap(f).flatMap(g) is equivalent to m.flatMap(x => f(x).flatMap(g)). Check.
Rebuttal to "Without substitution, laws break"
The meaning of equivalent in the monad laws, as I understand it, is you could replace one side of the expression with the other side in your code without changing the behavior of the program. Assuming you always use the same execution context, I think that is the case. In the example #sukant gave, it would have had the same issue if it had used Option instead of Future. I don't think the fact that the futures are evaluated eagerly is relevant.
As the other commenters have suggested, you are mistaken. Scala's Future type has the monadic properties:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits._
def unit[A](block: => A): Future[A] = Future(block)
def bind[A, B](fut: Future[A])(fun: A => Future[B]): Future[B] = fut.flatMap(fun)
This is why you can use for-comprehension syntax with futures in Scala.
I'm writing a programming language interpreter.
I have need of the right code idiom to both evaluate a sequence of expressions to get a sequence of their values, and propagate state from one evaluator to the next to the next as the evaluations take place. I'd like a functional programming idiom for this.
It's not a fold because the results come out like a map. It's not a map because of the state prop across.
What I have is this code which I'm using to try to figure this out. Bear with a few lines of test rig first:
// test rig
class MonadLearning extends JUnit3Suite {
val d = List("1", "2", "3") // some expressions to evaluate.
type ResType = Int
case class State(i : ResType) // trivial state for experiment purposes
val initialState = State(0)
// my stub/dummy "eval" function...obviously the real one will be...real.
def computeResultAndNewState(s : String, st : State) : (ResType, State) = {
val State(i) = st
val res = s.toInt + i
val newStateInt = i + 1
(res, State(newStateInt))
}
My current solution. Uses a var which is updated as the body of the map is evaluated:
def testTheVarWay() {
var state = initialState
val r = d.map {
s =>
{
val (result, newState) = computeResultAndNewState(s, state)
state = newState
result
}
}
println(r)
println(state)
}
I have what I consider unacceptable solutions using foldLeft which does what I call "bag it as you fold" idiom:
def testTheFoldWay() {
// This startFold thing, requires explicit type. That alone makes it muddy.
val startFold : (List[ResType], State) = (Nil, initialState)
val (r, state) = d.foldLeft(startFold) {
case ((tail, st), s) => {
val (r, ns) = computeResultAndNewState(s, st)
(tail :+ r, ns) // we want a constant-time append here, not O(N). Or could Cons on front and reverse later
}
}
println(r)
println(state)
}
I also have a couple of recursive variations (which are obvious, but also not clear or well motivated), one using streams which is almost tolerable:
def testTheStreamsWay() {
lazy val states = initialState #:: resultStates // there are states
lazy val args = d.toStream // there are arguments
lazy val argPairs = args zip states // put them together
lazy val resPairs : Stream[(ResType, State)] = argPairs.map{ case (d1, s1) => computeResultAndNewState(d1, s1) } // map across them
lazy val (results , resultStates) = myUnzip(resPairs)// Note .unzip causes infinite loop. Had to write my own.
lazy val r = results.toList
lazy val finalState = resultStates.last
println(r)
println(finalState)
}
But, I can't figure out anything as compact or clear as the original 'var' solution above, which I'm willing to live with, but I think somebody who eats/drinks/sleeps monad idioms is going to just say ... use this... (Hopefully!)
With the map-with-accumulator combinator (the easy way)
The higher-order function you want is mapAccumL. It's in Haskell's standard library, but for Scala you'll have to use something like Scalaz.
First the imports (note that I'm using Scalaz 7 here; for previous versions you'd import Scalaz._):
import scalaz._, syntax.std.list._
And then it's a one-liner:
scala> d.mapAccumLeft(initialState, computeResultAndNewState)
res1: (State, List[ResType]) = (State(3),List(1, 3, 5))
Note that I've had to reverse the order of your evaluator's arguments and the return value tuple to match the signatures expected by mapAccumLeft (state first in both cases).
With the state monad (the slightly less easy way)
As Petr Pudlák points out in another answer, you can also use the state monad to solve this problem. Scalaz actually provides a number of facilities that make working with the state monad much easier than the version in his answer suggests, and they won't fit in a comment, so I'm adding them here.
First of all, Scalaz does provide a mapM—it's just called traverse (which is a little more general, as Petr Pudlák notes in his comment). So assuming we've got the following (I'm using Scalaz 7 again here):
import scalaz._, Scalaz._
type ResType = Int
case class Container(i: ResType)
val initial = Container(0)
val d = List("1", "2", "3")
def compute(s: String): State[Container, ResType] = State {
case Container(i) => (Container(i + 1), s.toInt + i)
}
We can write this:
d.traverse[({type L[X] = State[Container, X]})#L, ResType](compute).run(initial)
If you don't like the ugly type lambda, you can get rid of it like this:
type ContainerState[X] = State[Container, X]
d.traverse[ContainerState, ResType](compute).run(initial)
But it gets even better! Scalaz 7 gives you a version of traverse that's specialized for the state monad:
scala> d.traverseS(compute).run(initial)
res2: (Container, List[ResType]) = (Container(3),List(1, 3, 5))
And as if that wasn't enough, there's even a version with the run built in:
scala> d.runTraverseS(initial)(compute)
res3: (Container, List[ResType]) = (Container(3),List(1, 3, 5))
Still not as nice as the mapAccumLeft version, in my opinion, but pretty clean.
What you're describing is a computation within the state monad. I believe that the answer to your question
It's not a fold because the results come out like a map. It's not a map because of the state prop across.
is that it's a monadic map using the state monad.
Values of the state monad are computations that read some internal state, possibly modify it, and return some value. It is often used in Haskell (see here or here).
For Scala, there is a trait in the ScalaZ library called State that models it (see also the source). There are utility methods in States for creating instances of State. Note that from the monadic point of view State is just a monadic value. This may seem confusing at first, because it's described by a function depending on a state. (A monadic function would be something of type A => State[B].)
Next you need is a monadic map function that computes values of your expressions, threading the state through the computations. In Haskell, there is a library method mapM that does just that, when specialized to the state monad.
In Scala, there is no such library function (if it is, please correct me). But it's possible to create one. To give a full example:
import scalaz._;
object StateExample
extends App
with States /* utility methods */
{
// The context that is threaded through the state.
// In our case, it just maps variables to integer values.
class Context(val map: Map[String,Int]);
// An example that returns the requested variable's value
// and increases it's value in the context.
def eval(expression: String): State[Context,Int] =
state((ctx: Context) => {
val v = ctx.map.get(expression).getOrElse(0);
(new Context(ctx.map + ((expression, v + 1)) ), v);
});
// Specialization of Haskell's mapM to our State monad.
def mapState[S,A,B](f: A => State[S,B])(xs: Seq[A]): State[S,Seq[B]] =
state((initState: S) => {
var s = initState;
// process the sequence, threading the state
// through the computation
val ys = for(x <- xs) yield { val r = f(x)(s); s = r._1; r._2 };
// return the final state and the output result
(s, ys);
});
// Example: Try to evaluate some variables, starting from an empty context.
val expressions = Seq("x", "y", "y", "x", "z", "x");
print( mapState(eval)(expressions) ! new Context(Map[String,Int]()) );
}
This way you can create simple functions that take some arguments and return State and then combine them into more complex ones by using State.map or State.flatMap (or perhaps better using for comprehensions), and then you can run the whole computation on a list of expressions by mapM.
See also http://blog.tmorris.net/posts/the-state-monad-for-scala-users/
Edit: See Travis Brown's answer, he described how to use the state monad in Scala much more nicely.
He also asks:
But why, when there's a standard combinator that does exactly what you need in this case?
(I ask this as someone who's been slapped for using the state monad when mapAccumL would do.)
It's because the original question asked:
It's not a fold because the results come out like a map. It's not a map because of the state prop across.
and I believe the proper answer is it is a monadic map using the state monad.
Using mapAccumL is surely faster, both less memory and CPU overhead. But the state monad captures the concept of what is going on, the essence of the problem. I believe in many (if not most) cases this is more important. Once we realize the essence of the problem, we can either use the high-level concepts to nicely describe the solution (perhaps sacrificing speed/memory a little) or optimize it to be fast (or perhaps even manage to do both).
On the other hand, mapAccumL solves this particular problem, but doesn't give us a broader answer. If we need to modify it a little, it might happen it won't work any more. Or, if the library starts to be complex, the code can start to be messy and we won't know how to improve it, how to make the original idea clear again.
For example, in the case of evaluating stateful expressions, the library can become complicated and complex. But if we use the state monad, we can build the library around small functions, each taking some arguments and returning something like State[Context,Result]. These atomic computations can be combined to more complex ones using flatMap method or for comprehensions, and finally we'll construct the desired task. The principle will stay the same across the whole library, and the final task will also be something that returns State[Context,Result].
To conclude: I'm not saying using the state monad is the best solution, and certainly it's not the fastest one. I just believe it is most didactic for a functional programmer - it describes the problem in a clean, abstract way.
You could do this recursively:
def testTheRecWay(xs: Seq[String]) = {
def innerTestTheRecWay(xs: Seq[String], priorState: State = initialState, result: Vector[ResType] = Vector()): Seq[ResType] = {
xs match {
case Nil => result
case x :: tail =>
val (res, newState) = computeResultAndNewState(x, priorState)
innerTestTheRecWay(tail, newState, result :+ res)
}
}
innerTestTheRecWay(xs)
}
Recursion is a common practice in functional programming and is most of the time easier to read, write and understand than loops. In fact scala does not have any loops other than while. fold, map, flatMap, for (which is just sugar for flatMap/map), etc. are all recursive.
This method is tail recursive and will be optimized by the compiler to not build a stack, so it is absolutely safe to use. You can add the #annotation.tailrec annotaion, to force the compiler to apply tail recursion elimination. If your method is not tailrec the compiler will then complain.
edit: renamed inner method to avoid ambiguity
How do you use scalaz.WriterT for logging?
About monad transformers
This is a very short introduction. You may find more information on haskellwiki or this great slide by #jrwest.
Monads don't compose, meaning that if you have a monad A[_] and a monad B[_], then A[B[_]] can not be derived automatically. However in most cases this can be achieved by having a so-called monad transformer for a given monad.
If we have monad transformer BT for monad B, then we can compose a new monad A[B[_]] for any monad A. That's right, by using BT, we can put the B inside A.
Monad transformer usage in scalaz
The following assumes scalaz 7, since frankly I didn't use monad transformers with scalaz 6.
A monad transformer MT takes two type parameters, the first is the wrapper (outside) monad, the second is the actual data type at the bottom of the monad stack. Note: It may take more type parameters, but those are not related to the transformer-ness, but rather specific for that given monad (like the logged type of a Writer, or the error type of a Validation).
So if we have a List[Option[A]] which we would like to treat as a single composed monad, then we need OptionT[List, A]. If we have Option[List[A]], we need ListT[Option, A].
How to get there? If we have the non-transformer value, we can usually just wrap it with MT.apply to get the value inside the transformer. To get back from the transformed form to normal, we usually call .run on the transformed value.
So val a: OptionT[List, Int] = OptionT[List, Int](List(some(1)) and val b: List[Option[Int]] = a.run are the same data, just the representation is different.
It was suggested by Tony Morris that is best to go into the transformed version as early as possible and use that as long as possible.
Note: Composing multiple monads using transformers yields a transformer stack with types just the opposite order as the normal data type. So a normal List[Option[Validation[E, A]]] would look something like type ListOptionValidation[+E, +A] = ValidationT[({type l[+a] = OptionT[List, a]})#l, E, A]
Update: As of scalaz 7.0.0-M2, Validation is (correctly) not a Monad and so ValidationT doesn't exist. Use EitherT instead.
Using WriterT for logging
Based on your need, you can use the WriterT without any particular outer monad (in this case in the background it will use the Id monad which doesn't do anything), or can put the logging inside a monad, or put a monad inside the logging.
First case, simple logging
import scalaz.{Writer}
import scalaz.std.list.listMonoid
import scalaz._
def calc1 = Writer(List("doing calc"), 11)
def calc2 = Writer(List("doing other"), 22)
val r = for {
a <- calc1
b <- calc2
} yield {
a + b
}
r.run should be_== (List("doing calc", "doing other"), 33)
We import the listMonoid instance, since it also provides the Semigroup[List] instance. It is needed since WriterT needs the log type to be a semigroup in order to be able to combine the log values.
Second case, logging inside a monad
Here we chose the Option monad for simplicity.
import scalaz.{Writer, WriterT}
import scalaz.std.list.listMonoid
import scalaz.std.option.optionInstance
import scalaz.syntax.pointed._
def calc1 = WriterT((List("doing calc") -> 11).point[Option])
def calc2 = WriterT((List("doing other") -> 22).point[Option])
val r = for {
a <- calc1
b <- calc2
} yield {
a + b
}
r.run should be_== (Some(List("doing calc", "doing other"), 33))
With this approach, since the logging is inside the Option monad, if any of the bound options is None, we would just get a None result without any logs.
Note: x.point[Option] is the same in effect as Some(x), but may help to generalize the code better. Not lethal just did it that way for now.
Third option, logging outside of a monad
import scalaz.{Writer, OptionT}
import scalaz.std.list.listMonoid
import scalaz.std.option.optionInstance
import scalaz.syntax.pointed._
type Logger[+A] = WriterT[scalaz.Id.Id, List[String], A]
def calc1 = OptionT[Logger, Int](Writer(List("doing calc"), Some(11): Option[Int]))
def calc2 = OptionT[Logger, Int](Writer(List("doing other"), None: Option[Int]))
val r = for {
a <- calc1
b <- calc2
} yield {
a + b
}
r.run.run should be_== (List("doing calc", "doing other") -> None)
Here we use OptionT to put the Option monad inside the Writer. One of the calculations is Noneto show that even in this case logs are preserved.
Final remarks
In these examples List[String] was used as the log type. However using String is hardly ever the best way, just some convention forced on us by logging frameworks. It would be better to define a custom log ADT for example, and if needed to output, convert it to string as late as possible. This way you could serialize the log's ADT and easily analyse it later programmatically (instead of parsing strings).
WriterT has a host of useful methods to work with to ease logging, check out the source. For example given a w: WriterT[...], you may add a new log entry using w :++> List("other event"), or even log using the currently held value using w :++>> ((v) => List("the result is " + v)), etc.
There are many explicit and longish code (types, calls) in the examples. As always, these are for clarity, refactor them in your code by extracting common types and ops.
type OptionLogger[A] = WriterT[Option, NonEmptyList[String], A]
val two: OptionLogger[Int] = WriterT.put(2.some)("The number two".pure[NonEmptyList])
val hundred: OptionLogger[Int] = WriterT.put(100.some)("One hundred".pure[NonEmptyList])
val twoHundred = for {
a <- two
b <- hundred
} yield a * b
twoHundred.value must be equalTo(200.some)
val log = twoHundred.written map { _.list } getOrElse List() mkString(" ")
log must be equalTo("The number two One hundred")