I'm trying to chain a few sequential operations in a functional way with Scala and Cats. They look perfect separately but I'm not sure how can I chain them now with a flatMap / for comprehension.
So, let's say, I have something like
import cats.data.State
object Step1 {
def apply() = State[String, Seq[String]] { text =>
val ans = text.trim.split("""[\s]+""").toSeq
(text, ans)
}
}
println(Step1().run("Lorem Ipsum Dolor").value)
object Step2 {
def apply() = State[Seq[String], Seq[String]] { terms =>
val ans = terms.map(_.toLowerCase)
(terms, ans)
}
}
println(Step2().run(Seq("Lorem", "Ipsum", "Dolor")).value)
Ideally, I'd like to have something like
for {
a <- Step1()
b <- Step2()
} yield (b)
What is the best way to achieve this?
Take note of your types:
For your Step1, you have State[String, Seq[String]].
For your Step2, you have State[Seq[String], Seq[String]].
The function flatMap takes in an argument of M[A] and A => M[B] and returns M[B] but clearly your M[_] for Step1 and Step2 are clearly different even though they are both using the State datatype.
Take note that State has a type signature of * -> * -> * or it looks something like State[S, A] where your S is your "state" and A is your value.
In this case, if you really want to flatMap the two distinct State then you have to first "adjust" and equate the S of one of them.
Related
I want to log in the event that a record doesn't have an adjoining record. Is there a purely functional way to do this? One that separates the side effect from the data transformation?
Here's an example of what I need to do:
val records: Seq[Record] = Seq(record1, record2, ...)
val accountsMap: Map[Long, Account] = Map(record1.id -> account1, ...)
def withAccount(accountsMap: Map[Long, Account])(r: Record): (Record, Option[Account]) = {
(r, accountsMap.get(r.id))
}
def handleNoAccounts(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
tuple
}
def toRichAccount(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
a.map(acct => RichAccount(r, acct))
}
records
.map(withAccount(accountsMap))
.map(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
So there are multiple issues with this approach that I think make it less than optimal.
The tuple return type is clumsy. I have to destructure the tuple in both of the latter two functions.
The logging function has to handle the logging and then return the tuple with no changes. It feels weird that this is passed to .map even though no transformation is taking place -- maybe there is a better way to get this side effect.
Is there a functional way to clean this up?
I could be wrong (I often am) but I think this does everything that's required.
records
.flatMap(r =>
accountsMap.get(r.id).fold{
logger.error(s"no account for ${r.id}")
Option.empty[RichAccount]
}{a => Some(RichAccount(r,a))})
If you're using scala 2.13 or newer you could use tapEach, which takes function A => Unit to apply side effect on every element of function and then passes collection unchanged:
//you no longer need to return tuple in side-effecting function
def handleNoAccounts(tuple: (Record, Option[Account]): Unit = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
}
records
.map(withAccount(accountsMap))
.tapEach(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
In case you're using older Scala, you could provide extension method (updated according to Levi's Ramsey suggestion):
implicit class SeqOps[A](s: Seq[A]) {
def tapEach(f: A => Unit): Seq[A] = {
s.foreach(f)
s
}
}
I am trying to use Cats datatype Ior to accumulate both errors and successes of using a service (which can return an error).
def find(key: String): F[Ior[NonEmptyList[Error], A]] = {
(for {
b <- service.findByKey(key)
} yield b.rightIor[NonEmptyList[Error]])
.recover {
case e: Error => Ior.leftNel(AnotherError)
}
}
def findMultiple(keys: List[String]): F[Ior[NonEmptyList[Error], List[A]]] = {
keys map find reduce (_ |+| _)
}
My confusion lies in how to combine the errors/successes. I am trying to use the Semigroup combine (infix syntax) to combine with no success. Is there a better way to do this? Any help would be great.
I'm going to assume that you want both all errors and all successful results. Here's a possible implementation:
class Foo[F[_]: Applicative, A](find: String => F[IorNel[Error, A]]) {
def findMultiple(keys: List[String]): F[IorNel[Error, List[A]]] = {
keys.map(find).sequence.map { nelsList =>
nelsList.map(nel => nel.map(List(_)))
.reduceOption(_ |+| _).getOrElse(Nil.rightIor)
}
}
}
Let's break it down:
We will be trying to "flip" a List[IorNel[Error, A]] into IorNel[Error, List[A]]. However, from doing keys.map(find) we get List[F[IorNel[...]]], so we need to also "flip" it in a similar fashion first. That can be done by using .sequence on the result, and is what forces F[_]: Applicative constraint.
N.B. Applicative[Future] is available whenever there's an implicit ExecutionContext in scope. You can also get rid of F and use Future.sequence directly.
Now, we have F[List[IorNel[Error, A]]], so we want to map the inner part to transform the nelsList we got. You might think that sequence could be used there too, but it can not - it has the "short-circuit on first error" behavior, so we'd lose all successful values. Let's try to use |+| instead.
Ior[X, Y] has a Semigroup instance when both X and Y have one. Since we're using IorNel, X = NonEmptyList[Z], and that is satisfied. For Y = A - your domain type - it might not be available.
But we don't want to combine all results into a single A, we want Y = List[A] (which also always has a semigroup). So, we take every IorNel[Error, A] we have and map A to a singleton List[A]:
nelsList.map(nel => nel.map(List(_)))
This gives us List[IorNel[Error, List[A]], which we can reduce. Unfortunately, since Ior does not have a Monoid, we can't quite use convenient syntax. So, with stdlib collections, one way is to do .reduceOption(_ |+| _).getOrElse(Nil.rightIor).
This can be improved by doing few things:
x.map(f).sequence is equivalent to doing x.traverse(f)
We can demand that keys are non-empty upfront, and give nonempty result back too.
The latter step gives us Reducible instance for a collection, letting us shorten everything by doing reduceMap
class Foo2[F[_]: Applicative, A](find: String => F[IorNel[Error, A]]) {
def findMultiple(keys: NonEmptyList[String]): F[IorNel[Error, NonEmptyList[A]]] = {
keys.traverse(find).map { nelsList =>
nelsList.reduceMap(nel => nel.map(NonEmptyList.one))
}
}
}
Of course, you can make a one-liner out of this:
keys.traverse(find).map(_.reduceMap(_.map(NonEmptyList.one)))
Or, you can do the non-emptiness check inside:
class Foo3[F[_]: Applicative, A](find: String => F[IorNel[Error, A]]) {
def findMultiple(keys: List[String]): F[IorNel[Error, List[A]]] = {
NonEmptyList.fromList(keys)
.map(_.traverse(find).map { _.reduceMap(_.map(List(_))) })
.getOrElse(List.empty[A].rightIor.pure[F])
}
}
Ior is a good choice for warning accumulation, that is errors and a successful value. But, as mentioned by Oleg Pyzhcov, Ior.Left case is short-circuiting. This example illustrates it:
scala> val shortCircuitingErrors = List(
Ior.leftNec("error1"),
Ior.bothNec("warning2", 2),
Ior.bothNec("warning3", 3)
).sequence
shortCircuitingErrors: Ior[Nec[String], List[Int]]] = Left(Chain(error1))
One way to accumulate both errors and successes is to convert all your Left cases into Both. One approach is using Option as right type and converting Left(errs) values into Both(errs, None). After calling .traverse, you end up with optList: List[Option] on the right side and you can flatten it with optList.flatMap(_.toList) to filter out None values.
class Error
class KeyValue
def find(key: String): Ior[Nel[Error], KeyValue] = ???
def findMultiple(keys: List[String]): Ior[Nel[Error], List[KeyValue]] =
keys
.traverse { k =>
val ior = find(k)
ior.putRight(ior.right)
}
.map(_.flatMap(_.toList))
Or more succinctly:
def findMultiple(keys: List[String]): Ior[Nel[Error], List[KeyValue]] =
keys.flatTraverse { k =>
val ior = find(k)
ior.putRight(ior.toList) // Ior[A,B].toList: List[B]
}
I often want to log or to print something without changing it.
It looks like this:
val result = myResult // this could be an Option or a Future
.map{r =>
info(s"the result is $r")
r
}
These three lines are always the same.
In a for comprehension, this can be done a bit nicer.
But I look for a solution for the first declarative version. It should look like:
val result = myResult
.log(info(s"the result is ${_}"))
This one-liner could be put in every place in the chain where there could be a map, like:
val result = myResult
.log(info(s"1. ${_}"))
.filter(_ > 1)
.log(info(s"2. ${_}"))
...
How could this be achieved? If possible, without a functional library.
Ok so I decided to take a swing at this and I would like to retract my comment of
Maybe you can define a implicit class that has a method "log" acting on a Product ?
I was confident that Future and all the monads (collections, options) shared a common ancestor, turns out I was wrong. I have the following solution without using Cats. This can be done in a much prettier way in cats, besides the aforementioned "flatTap", and embelished with possibly cats.Ref or something.
Future is the obvious outliner here but as more exceptions come along you might need to expand this object.
import scala.concurrent._
import ExecutionContext.Implicits.global
object MonadicConv {
implicit class MonadicLog[+B <: Product, A] (val u: B){
def log(l: String, args: List[A] = List()): B = {
println("logging")
println(u)
println(l)
u
}
}
implicit class FutureLog[T, A](val u: Future[T]){
def log(l: String, args: List[A] = List()) : Future[T] = {
println("logging")
println(u)
println(l)
u
}
}
}
1) You will need to modify this with you own logging logic, I am just printing
2) I am not super proud of this as this is no longer a pure function. I am not sure if there is a work around this in Scala without using Cats. (There might be)
3) The args can be removed, just added them in case you want to pass in extra info
4) If you really want to combine these, you could try defining your own product, some leads: Implement product type in Scala with generic update function working on its parts
You can use this with
import MonadicConv._
val x = Some(5).log("").get
val lx = List(Some(5), Some(10), Some(1)).log("list").flatten.log("x").filter(_ > 1).log("")
val ff = Future.successful(List(Some(5), Some(10), Some(1))).log("fff").map(_.flatten.filter(_ > 1).log("inner")).log("")
This prints out
logging
Some(5)
option test
logging
List(Some(5), Some(10), Some(1))
list test
logging
List(5, 10, 1)
flat test
logging
List(5, 10)
filter test
logging
Future(Success(List(Some(5), Some(10), Some(1))))
future test
logging
Future(<not completed>)
incomplete test
logging
List(5, 10)
inner future test
Scastie version here
As I have mentioned, this is really the Cats land at this point. This is the best I could come up with in core Scala
For your purpose, it is best to use treelog. It turns the logging process and values into a Monad of DescribedComputation :
import treelog.LogTreeSyntaxWithoutAnnotations._
val result: DescribedComputation[YourValueType] = myResult ~> (_.fold("The result is empty")(r => s"The result is $r")
And usually to subtract the value from a DescribedComputation, use for comprehension:
for {
res <- result
} {
doSomethingTo(res)
}
See details from https://github.com/lancewalton/treelog
The whole example will look like:
val compRes = "Logging Result" ~< {
for {
r <- myResult ~> (_.fold("The result is empty")(r => s"The result is $r")
} yield r
}
}
for (res <- compRes) {
doSomethingTo(res)
}
logger.info(logging.run.written.shows)
The output will look like:
2019-11-18 00:00:00,000 INFO Logging Result
The result is XXX
Just for reference. ZIO provides this functionality nicely.
/**
* Returns an effect that effectfully "peeks" at the success of this effect.
*
* {{{
* readFile("data.json").tap(putStrLn)
* }}}
*/
final def tap[R1 <: R, E1 >: E](f: A => ZIO[R1, E1, Any]): ZIO[R1, E1, A] = self.flatMap(new ZIO.TapFn(f))
There is even a version for the error case:
/**
* Returns an effect that effectfully "peeks" at the failure of this effect.
* {{{
* readFile("data.json").tapError(logError(_))
* }}}
*/
final def tapError[R1 <: R, E1 >: E](f: E => ZIO[R1, E1, Any]): ZIO[R1, E1, A]
This makes debugging really easy:
myDangerousZioFunction
.tapError(e => putStrLn(s"Server Exception: $e"))
.tap(r => putStrLn(s"Result is $r"))
....
I've got following problem:
val sth: Future[Seq[T, S]] = for {
x <- whatever: Future[List[T]]
y <- x: List[T]
z <- f(y): Future[Option[S]]
n <- z: Option[S]
} yield y: T -> n: S
I would want to make this code to work(I guess everyone understands the idea as I've added types).
By "to work" I mean, that I would want to stay with the for-comprehension structure and fulfil expected types in the end. I know there are "ugly" ways to do it, but I want to learn how to do it pure :)
As I read the internet I've reached the conclusion that my problem may be solved by the monad transformers & scalaz. Unfortunately, I couldn't find an example to help understand better how should I proceed.
At the moment I've tried scalaz and Eff monad libs, but I guess I still don't understand how it works because I couldn't solve my problem.
I will be grateful for any help.
EDIT: It supposed to be future of sequence, also regarding the "whatever" I get it as a parameter of the function, sorry for misleading you
You could do something like what you need using the scalaz ListT monad transformer
object Test {
import scalaz._
import ListT._
type T = String
type S = Int
val whatever: Future[List[T]] = ??? // you get this somewhere
def f(y: T): Future[Option[S]] = ??? // function that returns future of option
val sth: Future[List[(T, S)]] = (for {
y <- listT(whatever)
// you cannot mix list and option, but you can convert the option to a list of 1 item
n <- listT(f(y).map(_.toList))
} yield y -> n).run
}
N.B.: Since you start with a future, you cannot return a Seq[(T,S)], you can only have a future. You have to call Await.result if you want to block and get the result.
The problem with for comprehension is that it's not some kind of magic monadic "unwrapper", it's just a sequence of map, flatMap and filter.
As you may know map and flatMap operate only on "inner" type, leaving "outer" type of monad unchanged. This means you can't do this:
for {
x <- whatever: Future[List[T]]
y <- x: List[T]
} yield y
inside single for. Instead, you can do something like this:
for (x <- whatever: Future[List[T]])
yield for (y <- x: List[T]) yield y
Which looks kinda ugly.
Back to your case, I's easier to write whole transformation explicitly using map and flatMap, as it gives you greater visibility and control:
whatever.flatMap {
x: List[T] =>
Future.sequence(x.map {
y: T => f(y).map(y -> _)
}).map(_.collect {
case (y, Some(n)) => y -> n
})
}
Also, #trustnoone mentioned, you can't get rid of the Future without explicitly calling Await.
I'm trying to validate the parameters of a method for nullity but i don't find the solution...
Can someone tell me how to do?
I'm trying something like this:
def buildNormalCategory(user: User, parent: Category, name: String, description: String): Either[Error,Category] = {
val errors: Option[String] = for {
_ <- Option(user).toRight("User is mandatory for a normal category").right
_ <- Option(parent).toRight("Parent category is mandatory for a normal category").right
_ <- Option(name).toRight("Name is mandatory for a normal category").right
errors : Option[String] <- Option(description).toRight("Description is mandatory for a normal category").left.toOption
} yield errors
errors match {
case Some(errorString) => Left( Error(Error.FORBIDDEN,errorString) )
case None => Right( buildTrashCategory(user) )
}
}
If you're willing to use Scalaz, it has a handful of tools that make this kind of task more convenient, including a new Validation class and some useful right-biased type class instances for plain old scala.Either. I'll give an example of each here.
Accumulating errors with Validation
First for our Scalaz imports (note that we have to hide scalaz.Category to avoid the name conflict):
import scalaz.{ Category => _, _ }
import syntax.apply._, syntax.std.option._, syntax.validation._
I'm using Scalaz 7 for this example. You'd need to make some minor changes to use 6.
I'll assume we have this simplified model:
case class User(name: String)
case class Category(user: User, parent: Category, name: String, desc: String)
Next I'll define the following validation method, which you can easily adapt if you move to an approach that doesn't involve checking for null values:
def nonNull[A](a: A, msg: String): ValidationNel[String, A] =
Option(a).toSuccess(msg).toValidationNel
The Nel part stands for "non-empty list", and a ValidationNel[String, A] is essentially the same as an Either[List[String], A].
Now we use this method to check our arguments:
def buildCategory(user: User, parent: Category, name: String, desc: String) = (
nonNull(user, "User is mandatory for a normal category") |#|
nonNull(parent, "Parent category is mandatory for a normal category") |#|
nonNull(name, "Name is mandatory for a normal category") |#|
nonNull(desc, "Description is mandatory for a normal category")
)(Category.apply)
Note that Validation[Whatever, _] isn't a monad (for reasons discussed here, for example), but ValidationNel[String, _] is an applicative functor, and we're using that fact here when we "lift" Category.apply into it. See the appendix below for more information on applicative functors.
Now if we write something like this:
val result: ValidationNel[String, Category] =
buildCategory(User("mary"), null, null, "Some category.")
We'll get a failure with the accumulated errors:
Failure(
NonEmptyList(
Parent category is mandatory for a normal category,
Name is mandatory for a normal category
)
)
If all of the arguments had checked out, we'd have a Success with a Category value instead.
Failing fast with Either
One of the handy things about using applicative functors for validation is the ease with which you can swap out your approach to handling errors. If you want to fail on the first instead of accumulating them, you can essentially just change your nonNull method.
We do need a slightly different set of imports:
import scalaz.{ Category => _, _ }
import syntax.apply._, std.either._
But there's no need to change the case classes above.
Here's our new validation method:
def nonNull[A](a: A, msg: String): Either[String, A] = Option(a).toRight(msg)
Almost identical to the one above, except that we're using Either instead of ValidationNEL, and the default applicative functor instance that Scalaz provides for Either doesn't accumulate errors.
That's all we need to do to get the desired fail-fast behavior—no changes are necessary to our buildCategory method. Now if we write this:
val result: Either[String, Category] =
buildCategory(User("mary"), null, null, "Some category.")
The result will contain only the first error:
Left(Parent category is mandatory for a normal category)
Exactly as we wanted.
Appendix: Quick introduction to applicative functors
Suppose we have a method with a single argument:
def incremented(i: Int): Int = i + 1
And suppose also that we want to apply this method to some x: Option[Int] and get an Option[Int] back. The fact that Option is a functor and therefore provides a map method makes this easy:
val xi = x map incremented
We've "lifted" incremented into the Option functor; that is, we've essentially changed a function mapping Int to Int into one mapping Option[Int] to Option[Int] (although the syntax muddies that up a bit—the "lifting" metaphor is much clearer in a language like Haskell).
Now suppose we want to apply the following add method to x and y in a similar fashion.
def add(i: Int, j: Int): Int = i + j
val x: Option[Int] = users.find(_.name == "John").map(_.age)
val y: Option[Int] = users.find(_.name == "Mary").map(_.age) // Or whatever.
The fact that Option is a functor isn't enough. The fact that it's a monad, however, is, and we can use flatMap to get what we want:
val xy: Option[Int] = x.flatMap(xv => y.map(add(xv, _)))
Or, equivalently:
val xy: Option[Int] = for { xv <- x; yv <- y } yield add(xv, yv)
In a sense, though, the monadness of Option is overkill for this operation. There's a simpler abstraction—called an applicative functor—that's in-between a functor and a monad and that provides all the machinery we need.
Note that it's in-between in a formal sense: every monad is an applicative functor, every applicative functor is a functor, but not every applicative functor is a monad, etc.
Scalaz gives us an applicative functor instance for Option, so we can write the following:
import scalaz._, std.option._, syntax.apply._
val xy = (x |#| y)(add)
The syntax is a little odd, but the concept isn't any more complicated than the functor or monad examples above—we're just lifting add into the applicative functor. If we had a method f with three arguments, we could write the following:
val xyz = (x |#| y |#| z)(f)
And so on.
So why bother with applicative functors at all, when we've got monads? First of all, it's simply not possible to provide monad instances for some of the abstractions we want to work with—Validation is the perfect example.
Second (and relatedly), it's just a solid development practice to use the least powerful abstraction that will get the job done. In principle this may allow optimizations that wouldn't otherwise be possible, but more importantly it makes the code we write more reusable.
I completely support Ben James' suggestion to make a wrapper for the null-producing api. But you'll still have the same problem when writing that wrapper. So here are my suggestions.
Why monads why for comprehension? An overcomplication IMO. Here's how you could do that:
def buildNormalCategory
( user: User, parent: Category, name: String, description: String )
: Either[ Error, Category ]
= Either.cond(
!Seq(user, parent, name, description).contains(null),
buildTrashCategory(user),
Error(Error.FORBIDDEN, "null detected")
)
Or if you insist on having the error message store the name of the parameter, you could do the following, which would require a bit more boilerplate:
def buildNormalCategory
( user: User, parent: Category, name: String, description: String )
: Either[ Error, Category ]
= {
val nullParams
= Seq("user" -> user, "parent" -> parent,
"name" -> name, "description" -> description)
.collect{ case (n, null) => n }
Either.cond(
nullParams.isEmpty,
buildTrashCategory(user),
Error(
Error.FORBIDDEN,
"Null provided for the following parameters: " +
nullParams.mkString(", ")
)
)
}
If you like the applicative functor approach of #Travis Brown's answer, but you don't like the Scalaz syntax or otherwise just don't want to use Scalaz, here is a simple library which enriches the standard library Either class to act as an applicative functor validation: https://github.com/youdevise/eithervalidation
For example:
import com.youdevise.eithervalidation.EitherValidation.Implicits._
def buildNormalCategory(user: User, parent: Category, name: String, description: String): Either[List[Error], Category] = {
val validUser = Option(user).toRight(List("User is mandatory for a normal category"))
val validParent = Option(parent).toRight(List("Parent category is mandatory for a normal category"))
val validName = Option(name).toRight(List("Name is mandatory for a normal category"))
Right(Category)(validUser, validParent, validName).
left.map(_.map(errorString => Error(Error.FORBIDDEN, errorString)))
}
In other words, this function will return a Right containing your Category if all of the Eithers were Rights, or it will return a Left containing a List of all the Errors, if one or more were Lefts.
Notice the arguably more Scala-ish and less Haskell-ish syntax, and a smaller library ;)
Lets suppose you have completed Either with the following quick and dirty stuff:
object Validation {
var errors = List[String]()
implicit class Either2[X] (x: Either[String,X]){
def fmap[Y](f: X => Y) = {
errors = List[String]()
//println(s"errors are $errors")
x match {
case Left(s) => {errors = s :: errors ; Left(errors)}
case Right(x) => Right(f(x))
}
}
def fapply[Y](f: Either[List[String],X=>Y]) = {
x match {
case Left(s) => {errors = s :: errors ; Left(errors)}
case Right(v) => {
if (f.isLeft) Left(errors) else Right(f.right.get(v))
}
}
}
}}
consider a validation function returning an Either:
def whenNone (value: Option[String],msg:String): Either[String,String] =
if (value isEmpty) Left(msg) else Right(value.get)
an a curryfied constructor returning a tuple:
val me = ((user:String,parent:String,name:String)=> (user,parent,name)) curried
You can validate it with :
whenNone(None,"bad user")
.fapply(
whenNone(Some("parent"), "bad parent")
.fapply(
whenNone(None,"bad name")
.fmap(me )
))
Not a big deal.