In https://gist.github.com/satyagraha/897e427bfb5ed203e9d3054ac6705704 I have posted a Scala Cats validation scenario which seems reasonable, but I haven't found a very neat solution.
Essentially, there is a two-stage validation, where individual fields are validated, then a class constructor is called which may throw due to internal checks (in general this may not be under my control to change, hence the exception handling code). We wish to not to call the constructor if any field validation fails, but also combine any constructor failure into the final result. "Fail-fast" is definitely right here for the two-phase check.
This is a kind of flatMap problem, which the cats.data.Validated framework appears to handle via the cats.data.Validated#andThenoperation. However I couldn't find a particularly neat solution to the problem as you can see in the code. There are quite a limited number of operations available on a cats.syntax.CartesianBuilder and is wasn't clear to me how to link it with the andThen operation.
Any ideas welcome! Note there is a Cats issue https://github.com/typelevel/cats/issues/1343 which possibly is related, not sure.
For fail fast, chained validation it is easier to use Either than Validated. You can easily switch from Either to Validated or vice versa depending if you want error accumulation.
A possible solution to your problem would be to create a smart constructor for User which returns an Either[Message, User] and use this with Validated[Message, (Name, Date)].
import cats.implicits._
import cats.data.Validated
def user(name: Name, date: Date): Either[Message, User] =
Either.catchNonFatal(User(name, date)).leftMap(Message.toMessage)
// error accumulation -> Validated
val valids: Validated[Message, (Name, Date)] =
(validateName(nameRepr) |#| validateDate(dateDepr)).tupled
// error short circuiting -> either
val userOrMessage: Either[Message, User] =
valids.toEither.flatMap((user _).tupled)
// Either[Message,User] = Right(User(Name(joe),Date(now)))
I would make a helper second-order function to wrap the exception-throwing ones:
def attempt[A, B](f: A => B): A => Validated[Message, B] = a => tryNonFatal(f(a))
Also, default companions of case classes extend the FunctionN trait, so there's no need to do (User.apply _).tupled, it can be shortened to User.tupled (on custom companions, you need to write extends ((...) => ...)) but apply override will be autogenerated)
So we end up with that using andThen:
val valids = validateName(nameRepr) |#| validateDate(dateDepr)
val res: Validated[Message, User] = valids.tupled andThen attempt(User.tupled)
Related
I am trying to port a solution for DomainEventHandlers and -Dispatcher from PHP8 to Scala 3. Handlers should specify a list of events they can handle (in a type-safe way, preferably, by their classes). Handlers are dynamically registered with the Dispatcher, which should aggregate a map from the elements of the lists from each Handler to a List of Handlers for those events.
When an event is raised with the Dispatcher, it should check the class of the current event against the keys from the map, and pass the event to each Handler in each list of Handlers for a key if and only if the event's class is identical to or a subclass of the class specified by the key.
In dynamically typed OOP languages like PHP8, this is easy - a Handler stores a list of class-names, which can be reified simply by [ClassName]::class, then the Dispatcher gets the event's class via $event::class and performs an is_a-check for each HashMap-key, which checks both exact match and subclass-relationship.
In Scala 3, I can't seem to find a good way to do this. Working with underlying Java-reflections via getClass or Class[?] produces problems due to the mismatch between the Scala and Java type-systems (specifically, trailing $ being either present or not). In Scala 2, Tags would probably have been the way to go - but Scala 3 reflection is a different beast, and I have not found a way to utilize it to implement the above, and would appreciate advice.
Concretely, let's say we have
trait DomainEvent[D1 <: Serializable, D2 <: Serializable, A <: Aggregate]
extends Event[D1, D2]:
type AggregateType = A
val aggregateIdentifier: (String, UUID)
def applyAsPatch(aggregate: AggregateType): AggregateType
trait DomainEventHandler:
val handles: List[???]
def handle(event: DomainEvent[?, ?, ?]): ZIO[Any, Throwable, Unit]
object DomainEventDispatcher:
val registeredHandlers: scala.collection.mutable.Map[???, List[DomainEventHandler]] =
scala.collection.mutable.Map()
def registerHandler(handler: DomainEventHandler): Unit = ???
def raiseEvent(event: DomainEvent[?, ?, ?]): ZIO[Any, Throwable, Unit] = ???
I am unsure what to use in place of ??? in the DomainEventHandler's List and the Dispatcher's Map - the registerHandler and raiseEvent-implementations will follow from that.
Well, if your concrete event classes that you match aren't parametrized, it's pretty simple:
trait Event[A]
case class IntEvent(x: Int) extends Event[Int]
case class StringEvent(x: String) extends Event[String]
object Dispatcher {
var handlers = List.empty[PartialFunction[Event[_], String]]
def register(h: PartialFunction[Event[_], String]): Unit = { handlers = h :: handlers }
def dispatch(event: Event[_]) = handlers.flatMap { _.lift(event) }
}
Dispatcher.register { case e: IntEvent => s"Handled $e" }
Dispatcher.register {
case e: IntEvent => s"Handled ${e.x}"
case e: StringEvent => s"Handled ${e.x}"
}
Dispatcher.dispatch(new IntEvent()) // List(Handled 1, Handled IntEvent(1))
Dispatcher.dispatch(new StringEvent("foo")) // List(Handled foo)
But if you want to match on things like Event[Int], that makes things significantly more difficult. I wasn't able to find a good way to do it (though, I am by no means an expert in scala 3 features).
Not sure why they dropped ClassTag support ... I am taking it as a sign that matching on type parameters like this is no longer considered a good practice, and the "proper" solution to your problem is now naming all classes you want to match without type parameters.
I see a lot of F[Request[F]], F[Monitor[F]] and F[Something[F]] around. Although I understand the mechanics of how to handle this, I don't have an intuitive understanding of why some effect (F) should appear like this (both wrapping around Something and appearing inside that Something). Insights ?
A simple HTTP server can be represented with the following function:
Request => Response
However, we commonly need to perform an effectful operation such as retrieving data from database or call external service. Such cases we would define like this:
Request => F[Response]
http4s has own abstraction over Routes (Endpoints) called HttpRoutes[F] and it is type alias for Kleisli[OptionT[F, ?], Request, Response] as mentioned in the documentation.
As we know from cats documentation, Kleisli is just compositions of functions that return a monadic value like Option[A], Either[L, R], etc.
One of the most useful properties of functions is that they compose. That is, given a function A => B and a function B => C, we can combine them to create a new function A => C.
In this case http4s's Kleislei takes Request and returns F[Response], simply Request => F[Response]. F is an effect type.
With a bit of modification the above Request and Response types, we get the following:
type HttpRoutes[F] = Kleisli[OptionT[F, *], Request[F], Response[F]]
An HttpRoutes[F] is declared as a simple series of case statements. Each case statement attempts to match and optionally extract from an incoming Request[F]. The code associated with the first matching case is used to generate a F[Response[F]].
The simplest case statement matches all requests without extracting anything. The right hand side of the request must return a F[Response[F]].
We can see here simple demonstration of F[Response[F]]
scala> val getRoot = Request[IO](Method.GET, uri"/")
getRoot: org.http4s.Request[cats.effect.IO] = Request(method=GET, uri=/, headers=Headers())
scala> val io = service.orNotFound.run(getRoot)
io: cats.effect.IO[org.http4s.Response[cats.effect.IO]] = <function1>
Here cats.effect.IO[org.http4s.Response[cats.effect.IO]] is unresolved yet, this is aync result. It will be resolved in future at some point as simple Future/Promise. And we can enforce it to run:
scala> val response = io.unsafeRunSync
response: org.http4s.Response[cats.effect.IO] = Response(status=200, headers=Headers())
Kleisli
Http4s
Practical FP in Scala by Gabriel Volpe
F[_] is just an abstraction. It can be List or IO of something else with kind * -> *. The main idea that you can define common functions. A good example of this a Functor typeclass and the map function inside this class.
So in your code, you can say that you F is a functor (F[_]: Functor) and after it allows you to use a map function.
What would it be the best approach to solve this problem in the most functional (algebraic) way by using Scala and Cats (or maybe another library focused on Category Theory and/or functional programming)?
Resources
Provided we have the following methods which perform REST API calls to retrieve single pieces of information?
type FutureApiCallResult[A] = Future[Either[String, Option[A]]]
def getNameApiCall(id: Int): FutureApiCallResult[String]
def getAgeApiCall(id: Int): FutureApiCallResult[Int]
def getEmailApiCall(id: Int): FutureApiCallResult[String]
As you can see they produce asynchronous results. The Either monad is used to return possible errors during API calls and Option is used to return None whenever the resource is not found by the API (this case is not an error but a possible and desired result type).
Method to implement in a functional way
case class Person(name: String, age: Int, email: String)
def getPerson(id: Int): Future[Option[Person]] = ???
This method should used the three API calls methods defined above to asynchronously compose and return a Person or None if either any of the API calls failed or any of the API calls return None (the whole Person entity cannot be composed)
Requirements
For performance reasons all the API calls must be done in a parallel way
My guess
I think the best option would be to use the Cats Semigroupal Validated but I get lost when trying to deal with Future and so many nested Monads :S
Can anyone tell me how would you implement this (even if changing method signature or main concept) or point me to the right resources? Im quite new to Cats and Algebra in coding but I would like to learn how to handle this kind of situations so that I can use it at work.
The key requirement here is that it has to be done in parallel. It means that the obvious solution using a monad is out, because monadic bind is blocking (it needs the result in case it has to branch on it). So the best option is to use applicative.
I'm not a Scala programmer, so I can't show you the code, but the idea is that an applicative functor can lift functions of multiple arguments (a regular functor lifts functions of single argument using map). Here, you would use something like map3 to lift the three-argument constructor of Person to work on three FutureResults. A search for "applicative future in Scala" returns a few hits. There are also applicative instances for Either and Option and, unlike monads, applicatives can be composed together easily. Hope this helps.
You can make use of the cats.Parallel type class. This enables some really neat combinators with EitherT which when run in parallel will accumulate errors. So the easiest and most concise solution would be this:
type FutureResult[A] = EitherT[Future, NonEmptyList[String], Option[A]]
def getPerson(id: Int): FutureResult[Person] =
(getNameApiCall(id), getAgeApiCall(id), getEmailApiCall(id))
.parMapN((name, age, email) => (name, age, email).mapN(Person))
For more information on Parallel visit the cats documentation.
Edit: Here's another way without the inner Option:
type FutureResult[A] = EitherT[Future, NonEmptyList[String], A]
def getPerson(id: Int): FutureResult[Person] =
(getNameApiCall(id), getAgeApiCall(id), getEmailApiCall(id))
.parMapN(Person)
this is the only solution i came across with but still not satisfied because i have the feeling it could be done in a cleaner way
import cats.data.NonEmptyList
import cats.implicits._
import scala.concurrent.Future
case class Person(name: String, age: Int, email: String)
type FutureResult[A] = Future[Either[NonEmptyList[String], Option[A]]]
def getNameApiCall(id: Int): FutureResult[String] = ???
def getAgeApiCall(id: Int): FutureResult[Int] = ???
def getEmailApiCall(id: Int): FutureResult[String] = ???
def getPerson(id: Int): FutureResult[Person] =
(
getNameApiCall(id).map(_.toValidated),
getAgeApiCall(id).map(_.toValidated),
getEmailApiCall(id).map(_.toValidated)
).tupled // combine three futures
.map {
case (nameV, ageV, emailV) =>
(nameV, ageV, emailV).tupled // combine three Validated
.map(_.tupled) // combine three Options
.map(_.map { case (name, age, email) => Person(name, age, email) }) // wrap final result
}.map(_.toEither)
Personally I prefer to collapse all non-success conditions into the Future's failure. That really simplifies the error handling, like:
val futurePerson = for {
name <- getNameApiCall(id)
age <- getAgeApiCall(id)
email <- getEmailApiCall(id)
} yield Person(name, age, email)
futurePerson.recover {
case e: SomeKindOfError => ???
case e: AnotherKindOfError => ???
}
Note that this won't run the requests in parallel, to do so you'd need to move the future's creation outside of the for comprehension, like:
val futureName = getNameApiCall(id)
val futureAge = getAgeApiCall(id)
val futureEmail = getEmailApiCall(id)
val futurePerson = for {
name <- futureName
age <- futureAge
email <- futureEmail
} yield Person(name, age, email)
Using Scala, Play Framework, Slick 3, Specs2.
I have a repository layer and a service layer. The repositories are quite dumb, and I use specs2 to make sure the service layer does its job.
My repositories used to return futures, like this:
def findById(id: Long): Future[Option[Foo]] =
db.run(fooQuery(id))
Then it would be used in the service:
def foonicate(id: Long): Future[Foo] = {
fooRepository.findById(id).flatMap { optFoo =>
val foo: Foo = optFoo match {
case Some(foo) => [business logic returning Foo]
case None => [business logic returning Foo]
}
fooRepository.save(foo)
}
}
Services were easy to spec. In the service spec, the FooRepository would be mocked like this:
fooRepository.findById(3).returns(Future(Foo(3)))
I recently found the need for database transactions. Several queries should be combined into a single transaction. The prevailing opinion seems to be that it's perfectly ok to handle transaction logic in the service layer.
With that in mind, I've changed my repositories to return slick.dbio.DBIO and added a helper method to run the query transactionally:
def findById(id: Long): DBIO[Option[Foo]] =
fooQuery(id)
def run[T](query: DBIO[T]): Future[T] =
db.run(query.transactionally)
The services compose the DBIOs and finally call the repository to run the query:
def foonicate(id: Long): Future[Foo] = {
val query = fooRepository.findById(id).flatMap { optFoo =>
val foo: Foo = optFoo match {
case Some(foo) => [business logic finally returning Foo]
case None => [business logic finally returning Foo]
}
fooRepository.save(foo)
}
fooRepository.run(query)
}
That seems to work, but now I can only spec it like this:
val findFooDbio = DBIO.successful(None))
val saveFooDbio = DBIO.successful(Foo(3))
fooRepository.findById(3) returns findFooDbio
fooRepository.save(Foo(3)) returns saveFooDbio
fooRepository.run(any[DBIO[Foo]]) returns Future(Foo(3))
That any in the run mock sucks! Now I'm not testing the actual logic but instead accept any DBIO[Foo]! I've tried to use the following:
fooRepository.run(findFooDbio.flatMap(_ => saveFooDbio)) returns Future(Foo(3))
But it breaks with java.lang.NullPointerException: null, which is specs2's way of saying "sorry mate, the method with this parameter wasn't found". I've tried various variations, but none of them worked.
I suspect that might be because functions can't be compared:
scala> val a: Int => String = x => "hi"
a: Int => String = <function1>
scala> val b: Int => String = x => "hi"
b: Int => String = <function1>
scala> a == b
res1: Boolean = false
Any ideas how to spec DBIO composition without cheating?
I had a similar idea and also investigated it to see if I could:
create the same DBIO composition
use it with matcher in mocking
However I found out that it is actually infeasible in practice:
as you noticed, you cannot compare functions
additionally, when you investigate internals of DBIO, it is basically Free-monad-like structure - it has implementation for plain values and directly generated queries (then, if you extract the statements you could compare some part of the query), but there are also mappings which stores functions
even if you somehow managed to reuse functions, so that they had reference equality, implementations of DBIO do not care about overriding equals, so they would be different beast anyway
So knowing that, I gave up the initial idea. What I can recommend instead:
mock on any input of database.run - it is more error prone, as it won't notify you if test expectations start differing from returned results, but it's better than nothing
replace DBIO by some intermediate structure, that you know you can compare safely. E.g. Cats' Free monad implementation uses case classes so as long as you manage to ensure that functions are somehow comparable (e.g. by not creating them ad hoc, but instead using vals and objects), you could compare on intermediate representation, and mock whole interpret -> run process
replace unit tests with mocked database with integration tests with an actual database
try out Typed Tagless Final Interpreter pattern for handling databases - and basically inject in tests different monad than in production (e.g. prod -> service returning DBIO, production -> service returning Futures you want)
Actually, you could try many other things with Free, TTFI and swapping implementations. The bottom line is - you cannot reliably compare on DBIO, so design your code in a way, that you could test without doing so. It's not a pleasant answer, especially if you just wanted to put together test, and move on, but AFAIK there is no other way.
Given an Option, what is the idiomatic way to get its value or throw an exception trying?
def foo() : String = {
val x : Option[String] = ...
x.getOrException()
}
A throw "statement" is really an expression in Scala, and it has type Nothing, which is a subtype of every other type. This means you can just use plain old getOrElse:
def myGet[A](oa: Option[A]) = oa.getOrElse(throw new RuntimeException("Can't."))
You really, really shouldn't be doing this, though.
(EDIT: this is not the best or most idiomatic way to do it. I wrote it when I was not familiar with Scala. I leave it here for an example of how not to do it. Nowadays I would do as #TravisBrown)
I think it really boils down to two things:
how sure are you that the value is there?
how do you want to react if it isn't?
If at that point in your code you expect the value to be there, and in the remote case that it isn't you want your program to fail fast, then I would only do a normal get and let Scala throw a NoSuchElementException if there was no value:
def foo() : String = {
val x : Option[String] = ...
x.get
}
If you want to handle the case differently (throw your own exception) I think a more elegant way would look like this:
def foo(): String = {
val x: Option[String] = None
x match {
case Some(value) => value
case None => throw new MyRuntimeException("blah")
}
}
And of course if you want to supply your own alternative value for the case that the Option is None you would just use getOrElse:
def foo(): String = {
val x: Option[String] = None
x.getOrElse("my alternative value")
}
I hope this will help you to understand how to represent errors (and generally effects) using types.
Error handling strategies in functional Scala
Use Option to return optional values. For example - fail to find entity in storage.
Use Option(possiblyNull) to avoid instances of Some(null).
Use Either[Error, T] to report expected failure. For example - email format is wrong, cannot parse a string to a number, etc.
Model your errors as ADTs (simply speaking kind of type hierarchies) to use it, for example, on the Left of the Either to represent more complex error scenarios.
Throw Exception only to signal unexpected and not-recoverable failures. Like missing config file.
Use Either.catchOnly or Try or Cats.IO (advanced) rather than a catch block for handling unexpected failures. Hint: You can still use ADT but extend them from throwables. More about Either vs Try.
Use Validated data-type from Cats lib to accumulate errors rather than fail-fast (Either), but prefer Either's on module-level to simplify the composition of the program (to have the same types). For example - form data validation, parsing errors accumulation.
Use mentioned types and don't optimize program preemptively - since most probably, bottle-necks would be in business logic, not in effect types.
Such an approach will simplify maintenance and updates of your code since you can reason about it without going to implementation specifics (aka local-reasoning). Also - reduce bugs - you cannot miss an error in the type. And compose the program easier (with help of map, flatMap and other combinators) - since it's simpler on type level, rather than with non-local exceptions and side-effects.
More about learning functional Scala.
But be aware that sometimes with this approach types could stack up and it could become harder to compose things. Given, for example: x: Future[Either[Error, Option[T]]] What you can do:
Use map and flatMap in combination with pattern-matching to compose different values of such types, for example:
x.faltMap { case Right(Some(v)) => anotherFuture(v); case Left(er) => ... }
If it doesn't help you can try to use MonadTransformers (don't be scared of the name, it's just wrappers around the effect types like Either and Future)
Also, an option is to simplify your errors ADT by extending them from the Throwable to unify it with Future, then it'll be Future[Option[T]]
And finally, in your case one option will be:
def foo() : Either[Error, String] = {
val x : Option[String] = ...
x match {
case Some(v) => Right(v)
case None => Left(Error(reason))
}
}
Just use the .get method.
def get[T](o:Option[T]) = o.get
It will throw a NoSuchElementException if o is an instance of None.
Basically, I would work with options like this:
def addPrint(oi:Option[Int]) = oi.map(_+1).foreach(println)
addPrint(Some(41))
addPrint(Some(1336))
addPrint(None)
to avoid your specific question.
Scala now support this operation on maps using getOrElse() method, see documentation here
As pointed out already, throwing an exception in Scala is an expression as well.
So you can do the following:
myMap.getOrElse(myKey, throw new MyCustomException("Custom Message HERE")