Being writing a completely async library to access a remote service (using Play2.0), I'm using Promise and Validation to create non-blocking call, which has a type presenting fail and valid result at once.
Promise comes from Play2-scala, where Validation comes from scalaz.
So here is the type of examples of such functions
f :: A => Promise[Validation[E, B]]
g :: B => Promise[Validation[E, C]]
So far, so good, now if I want to compose them, I can simple use the fact that Promise present a flatMap, so I can do it with a for-comprehension
for (
x <- f(a);
y <- g(b)
) yield y
Ok, I took a shortcut to my problem here because I didn't reused the Validation results within the for-comprehension. So if I want to reuse x in g, here is how I could do
for (
x <- f(a); // x is a Validation
y <- x.fold(
fail => Promise.pure(x),
ok => g(ok)
)
) yield y
Fair enough, but this kind of boilerplate will go to pollute my code over and over again. The problem here is that I've a kind of two-levels Monadic structure like M[N[_]].
At this stage, is there any structure in f° programming that enables working with such structure by skipping easily the secong level:
for (
x <- f(a); //x is a B
y <- g(b)
) yield y
Now, below is how I achieved something similar.
I created kind of Monadic structure that wraps the two level in one, let say ValidationPromised which pimped the Promise type with two methods:
def /~> [EE >: E, B](f: Validation[E, A] => ValidationPromised[EE, B]): ValidationPromised[EE, B] =
promised flatMap { valid =>
f(valid).promised
}
def /~~>[EE >: E, B](f: A => ValidationPromised[EE, B]): ValidationPromised[EE, B] =
promised flatMap { valid =>
valid.fold (
bad => Promise.pure(KO(bad)),
good => f(good).promised
)
}
This allows me to do such things
endPoint.service /~~> //get the service
(svc => //the service
svc.start /~~> (st => //get the starting elt
svc.create(None) /~~> //svc creates a new elt
(newE => //the created one
newEntry.link(st, newE) /~~> //link start and the new
(lnk => Promise.pure(OK((st, lnk, newE)))) //returns a triple => hackish
)
)
)
As we can see /~~> is pretty similar to flatMap but skips one level. The problem is the verbosity (that's why "for-comprehension" exists in Scala and "do" in Haskell).
Another point, I've the /~> that stands like a map also but works on the second level (instead of the Valid type -- third level)
So my second question is corollary to the former... Am I approching a sustainable solution with this construction ?
sorry to be that long
The concept you are looking for here is monad transformers. In brief, monad transformers compensate for monads not composing by allowing you to "stack" them.
You didn't mention the version of Scalaz you are using, but if you look in the scalaz-seven branch, you'll find ValidationT. This can be used to wrap any F[Validation[E, A]] into a ValidationT[F, E, A], where in your case F = Promise. If you change f and g to return ValidationT, then you can leave your code as
for {
x ← f(a)
y ← g(b)
} yield y
This will give you a ValidationT[Promise, E, B] as a result.
Related
Title says it all, really; iterating over collection while preserving state between loops and finishing iteration based on termination condition in addition to simply running out of elements may be the most common pattern to accomplish anything in imperative programming. It seems to me however like it's something functional gentleprogrammers agreed to not talk about, or at least I never encountered an idiom for it or a semi-standarized name such as with map, fold, reduce, etc.
I often use the followinig code in scala:
implicit class FoldWhile[T](private val items :Iterable[T]) extends AnyVal {
def foldWhile[A](start :A)(until :A=>Boolean)(op :(A, T)=>A) :A = {
if (until(start)) start
else {
var accumulator = start
items.find{ e => accumulator = op(accumulator, e); until(accumulator) }
accumulator
}
}
}
But it's ugly. Whenever I try a more declarative approach, I come with even longer and almost surely slower code, akin to:
Iterator.iterate((start, items.iterator)){
case (acc, i) if until(acc) => (acc, i)
case (acc, i) if i.hasNext => (op(acc, i.next()), i)
case x => x
}.dropWhile {
case (acc, i) => !until(acc) && i.hasNext
}.next()._1
(A more functional variant would use Lists or Streams, but iterators have arguably lesser overhead than converting items to a Stream, as default implementation for the latter uses an iterator underneath anyway).
My questions are:
1) Does this concept have a name in functional programming, and if so, what is the pattern associated with its implementation?
2) What would be the best (i.e. concise, generic, lazy, and with least overhead) way to implememnt it in scala?
This is frowned upon by scala purists, but you can use a return statement like this:
def foldWhile[A](zero: A)(until:A => Boolean)(op: (A,T) => A): A = items.fold(zero) {
case (a, b) if until(a) => return a
case (a,b) => op(a, b)
}
Or, if you are one of those frowning, and would like a purely functional solution without dirty imperative tricks, you can use something lazy, like an iterator or a stream:
items
.toStream // or .iterator - it doesn't really matter much in this case
.scanLeft(zero)(op)
.find(until)
The functional way of doing such things is via Tail Recursion:
implicit class FoldWhile[T](val items: Iterable[T]) extends AnyVal {
def foldWhile[A](zero: A)(until: A => Boolean)(op: (A, T) => A): A = {
#tailrec def loop(acc: A, remaining: Iterable[T]): A =
if (remaining.isEmpty || !until(acc)) acc else loop(op(acc, remaining.head), remaining.tail)
loop(zero, items)
}
}
Using recursion you can decide at each step if you want to proceed or not without using break and without any overhead, because tail recursions are converted to iterations from the compiler.
Also, pattern matching is often used to decompose sequences. For example, if you had a List you could do:
implicit class FoldWhile[T](val items: List[T]) extends AnyVal {
def foldWhile[A](zero: A)(until: A => Boolean)(op: (A, T) => A): A = {
#tailrec def loop(acc: A, remaining: List[T]): A = remaining match {
case Nil => acc
case _ if !until(acc) => acc
case h :: t => loop(op(acc, h), t)
}
loop(zero, items)
}
}
Scala has the #scala.annotation.tailrec annotation to force compilation to fail if the function you're annotating is not tail recursive. I suggest you use it as much as you can because it helps both to avoid errors and document the code.
The functional name for this is an Iteratee.
There are a bunch of references about this, but it's probably better to start from where the design ended up by reading the Pipes Tutorial and only if you're interested working backwards from there to see how it came from an early terminating left fold.
A right fold, when done lazily, can do early termination. In Haskell, for example, you can write the find function (return first element of list that satisfies predicate) with foldr:
find :: (a -> Bool) -> [a] -> Maybe a
find p = foldr (\a r -> if p a then Just a else r) Nothing
-- For reference:
foldr :: (a -> r -> r) -> r -> [a] -> r
foldr _ z [] = []
foldr f z (a:as) = f a (foldr f z as)
What happens when you try, say, find even [1..]? (Note that this is an infinite list!)
find even [1..]
= foldr (\a r -> if even a then Just a else r) Nothing [1..]
= if even 1
then Just 1
else foldr (\a r -> if even a then Just a else r) Nothing ([2..])
= if False
then Just 1
else foldr (\a r -> if even a then Just a else r) Nothing ([2..])
= foldr (\a r -> if even a then Just a else r) Nothing ([2..])
= if even 2
then Just 2
else foldr (\a r -> if even a then Just a else r) Nothing ([3..])
= if True
then Just 2
else foldr (\a r -> if even a then Just a else r) Nothing ([3..])
= Just 2
Laziness means that the function that we fold with (\a r -> if even a then Just a else r) gets to decide whether to force the r argument—the one whose evaluation requires us to recurse down the list—at all. So when even 2 evaluates to True, we pick the branch of the if ... then ... else ... that discards the result computed off the tail of the list—which means we never evaluate it. (It also runs in constant space as well. While programmers in eager functional languages learn to avoid foldr because of space and termination issues, those aren't always true in lazy languages!)
This of course hinges on the fact that Haskell is lazily evaluated, but it should nevertheless be possible to simulate this in an eager language like Scala—I do know it has a lazy val feature that might be usable for this. It looks like you'd need to write a lazyFold function that does a right fold, but the recursion happens inside a lazy value. You might still have problems with space usage, though.
I've got following problem:
val sth: Future[Seq[T, S]] = for {
x <- whatever: Future[List[T]]
y <- x: List[T]
z <- f(y): Future[Option[S]]
n <- z: Option[S]
} yield y: T -> n: S
I would want to make this code to work(I guess everyone understands the idea as I've added types).
By "to work" I mean, that I would want to stay with the for-comprehension structure and fulfil expected types in the end. I know there are "ugly" ways to do it, but I want to learn how to do it pure :)
As I read the internet I've reached the conclusion that my problem may be solved by the monad transformers & scalaz. Unfortunately, I couldn't find an example to help understand better how should I proceed.
At the moment I've tried scalaz and Eff monad libs, but I guess I still don't understand how it works because I couldn't solve my problem.
I will be grateful for any help.
EDIT: It supposed to be future of sequence, also regarding the "whatever" I get it as a parameter of the function, sorry for misleading you
You could do something like what you need using the scalaz ListT monad transformer
object Test {
import scalaz._
import ListT._
type T = String
type S = Int
val whatever: Future[List[T]] = ??? // you get this somewhere
def f(y: T): Future[Option[S]] = ??? // function that returns future of option
val sth: Future[List[(T, S)]] = (for {
y <- listT(whatever)
// you cannot mix list and option, but you can convert the option to a list of 1 item
n <- listT(f(y).map(_.toList))
} yield y -> n).run
}
N.B.: Since you start with a future, you cannot return a Seq[(T,S)], you can only have a future. You have to call Await.result if you want to block and get the result.
The problem with for comprehension is that it's not some kind of magic monadic "unwrapper", it's just a sequence of map, flatMap and filter.
As you may know map and flatMap operate only on "inner" type, leaving "outer" type of monad unchanged. This means you can't do this:
for {
x <- whatever: Future[List[T]]
y <- x: List[T]
} yield y
inside single for. Instead, you can do something like this:
for (x <- whatever: Future[List[T]])
yield for (y <- x: List[T]) yield y
Which looks kinda ugly.
Back to your case, I's easier to write whole transformation explicitly using map and flatMap, as it gives you greater visibility and control:
whatever.flatMap {
x: List[T] =>
Future.sequence(x.map {
y: T => f(y).map(y -> _)
}).map(_.collect {
case (y, Some(n)) => y -> n
})
}
Also, #trustnoone mentioned, you can't get rid of the Future without explicitly calling Await.
This is a follow-up to my previous question
Suppose I have two validating functions that return either the input if it is valid or the error messages if it is not.
type Status[A] = ValidationNel[String, A]
val isPositive: Int => Status[Int] =
x => if (x > 0) x.success else s"$x not positive".failureNel
val isEven: Int => Status[Int] =
x => if (x % 2 == 0) x.success else s"$x not even".failureNel
Suppose also that I need to validate an instance of case class X:
case class X(x1: Int, // should be positive
x2: Int) // should be even
More specifically I need a function checkX: X => Status[X]. Moreover, I'd like to write checkX as a composition of isPositive and isEven.
val checkX: X => Status[X] =
({x => isPositive(x.x1)} |#| {x => isEven(x.x2)}) ((X.apply _).lift[Status])
Does it make sense ?
How would you write checkX as a composition of isPositive and isEven?
There are lots of ways to write this, but I like the following:
val checkX: X => Status[X] = x => isPositive(x.x1).tuple(isEven(x.x2)).as(x)
Or:
val checkX: X => Status[X] =
x => isPositive(x.x1) *> isEven(x.x2) *> x.point[Status]
The key point is that you want to run the two validations only for their "effects" and then return the original value in the new context. This is a perfectly legitimate applicative operation, as your own implementation shows. There are just some slightly nicer ways to write it.
It doesn't make sense, because the two functions can't really be composed. You'd (presumably) like to check whether x is positive and whether x is even - but in the case where it is both negative and odd, you'd like to receive both errors. But that can't ever happen as a composition of those two functions - once you apply either of the failing cases, you don't have an x any more to pass to the second function.
In my experience Validation is almost never the correct type to use, for this very reason.
If you want "fail-fast" behaviour where the result is either success or the first error, you should use \/ (i.e. type Status[A] = String \/ A in this case). If you want "accumulate all the error messages alongside the value" behaviour, you want Writer, i.e. type Status[A] = Writer[Vector[String], A]. Both of these types allow easy composition (because they have monad instances available) using e.g. Kleisli: Kleisli(isPositive) >==> isEven would work for either of these definitions of Status (but not for yours).
I've been tasked with attaching an audit trail onto a bunch of calcuations for reconstruction of values after the fact (i.e. people with business domain knowledge to decipher what went wrong.) The current code looks something like this:
def doSomething = f(x) orElse g(x,y,z) orElse h(p,q,r) orElse default
Each of these returns an Option. The new code should return a tuple of (Option, Audit.)
I've implemented it as
def doSomething = f(x) match{
case None => g_prime(x,y,z)
case x # Some(_) => (x, SomeAuditObject)
}
//and taking some liberties with the actual signature...
def g_prime(x,y,z) = g(x,y,z) match{
and so on until the "default." Each function chains to the next and the next and so on. I don't like it. It feels way too imperative. I'm missing something. There's some way of thinking about this problem that I'm just not seeing. Other than wrapping the return values into another Option, what is it?
You can use Monads to compose transformations that leave an audit trail. You can compose the audits inside the Monad. Have a look at this answer for further details.
I tried to produce an example for you. I did not know how to handle the final step of the for-comprehension which is a map and provides no audit trail. If you disallow the use of map you cannot use for-comprehensions but have to use plain calls to flatMap.
case class WithAudit[A](value: A, audit: String){
def flatMap[B](f: A => WithAudit[B]): WithAudit[B] = {
val bWithAudit = f(value)
WithAudit(bWithAudit.value, audit + ":" + bWithAudit.audit)
}
def map[B](f: A => B): WithAudit[B] = {
WithAudit(f(value), audit + ":applied unknown function")
}
}
def doSomething(in: Option[Int]): WithAudit[Option[Int]] = WithAudit(
in.map(x => x - 23),
"substract 23"
)
def somethingElse(in: Int): WithAudit[String] = WithAudit(
in.toString,
"convert to String"
)
val processed = for(
v <- WithAudit(Some(42), "input Some(42)");
proc <- doSomething(v);
intVal <- WithAudit(proc.getOrElse(0), "if invalid, insert default 0");
asString <- somethingElse(intVal)
) yield asString
println(processed)
The output will be
WithAudit(
19,
input Some(42)
:substract 23
:if invalid, insert default 0
:convert to String
:applied unknown function
)
Safety
Using flatMap to process the value enforces the provision of an audit. If you don't provide map and limit how you can extract the value from the monad (maybe write a log output if you do so) you can be pretty safely assume that every transformation on the value will get logged. And when the value is obtained, you'll get an entry in your log.
Do you only audit successful executions of f, g, etc.?
If it is so, I'd make doSomething return Option[(YourData, Audit)] (instead of (Option[YourData], Audit)). You could then compose the functions like this:
def doSomething = (f(x) andThen (t => (t, audit_f(x)))) orElse
(g(x, y, z) andThen (t => (t, audit_g(x, y, z)))) orElse
... etc.
This question isn't meant as flame-bait! As it might be apparent, I've been looking at Scalaz recently. I'm trying to understand why I need some of the functionality that the library provides. Here's something:
import scalaz._
import Scalaz._
type NEL[A] = NonEmptyList[A]
val NEL = NonEmptyList
I put some println statements in my functions to see what was going on (aside: what would I have done if I was trying to avoid side effects like that?). My functions are:
val f: NEL[Int] => String = (l: NEL[Int]) => {println("f: " + l); l.toString |+| "X" }
val g: NEL[String] => BigInt = (l: NEL[String]) => {println("g: " + l); BigInt(l.map(_.length).sum) }
Then I combine them via a cokleisli and pass in a NEL[Int]
val k = cokleisli(f) =>= cokleisli(g)
println("RES: " + k( NEL(1, 2, 3) ))
What does this print?
f: NonEmptyList(1, 2, 3)
f: NonEmptyList(2, 3)
f: NonEmptyList(3)
g: NonEmptyList(NonEmptyList(1, 2, 3)X, NonEmptyList(2, 3)X, NonEmptyList(3)X)
RES: 57
The RES value is the character count of the (String) elements in the final NEL. Two things occur to me:
How could I have known that my NEL was going to be reduced in this manner from the method signatures involved? (I wasn't expecting the result at all)
What is the point of this? Can a reasonably simple and easy-to-follow use case be distilled for me?
This question is a thinly-veiled plea for some lovely person like retronym to explain how this powerful library actually works.
To understand the result, you need to understand the Comonad[NonEmptyList] instance. Comonad[W] essentially provides three functions (the actual interface in Scalaz is a little different, but this helps with explanation):
map: (A => B) => W[A] => W[B]
copure: W[A] => A
cojoin: W[A] => W[W[A]]
So, Comonad provides an interface for some container W that has a distinguished "head" element (copure) and a way of exposing the inner structure of the container so that we get one container per element (cojoin), each with a given element at the head.
The way that this is implemented for NonEmptyList is that copure returns the head of the list, and cojoin returns a list of lists, with this list at the head and all tails of this list at the tail.
Example (I'm shortening NonEmptyList to Nel):
Nel(1,2,3).copure = 1
Nel(1,2,3).cojoin = Nel(Nel(1,2,3),Nel(2,3),Nel(3))
The =>= function is coKleisli composition. How would you compose two functions f: W[A] => B and g: W[B] => C, knowing nothing about them other than that W is a Comonad? The input type of f and the output type of g aren't compatible. However, you can map(f) to get W[W[A]] => W[B] and then compose that with g. Now, given a W[A], you can cojoin it to get the W[W[A]] to feed into that function. So, the only reasonable composition is a function k that does the following:
k(x) = g(x.cojoin.map(f))
So for your nonempty list:
g(Nel(1,2,3).cojoin.map(f))
= g(Nel(Nel(1,2,3),Nel(2,3),Nel(3)).map(f))
= g(Nel("Nel(1,2,3)X","Nel(2,3)X","Nel(3)X"))
= BigInt(Nel("Nel(1,2,3)X","Nel(2,3)X","Nel(3)X").map(_.length).sum)
= BigInt(Nel(11,9,7).sum)
= 27
Cojoin is also defined for scalaz.Tree and scalaz.TreeLoc. This can be exploited to find a stream of all paths from the root of the tree to each leaf node.
def leafPaths[T](tree: Tree[T]): Stream[Stream[T]]
= tree.loc.cojoin.toTree.flatten.filter(_.isLeaf).map(_.path)
Using coKleisli arrow composition, we can do this, for example:
def leafDist[A] = (cokleisli(leafPaths[A]) &&& cokleisli(_.rootLabel))
=>= (_.map(s => (s._2, s._1.map(_.length).max)))
leafDist takes a Tree and returns a copy of it with each node annotated with its maximum distance from a leaf.