I've been struggling with error accumulation with effects, and I was wondering what the idiomatic way of modeling this is using Cats Effect.
Without involving effects, I would do something like this:
import cats._
import cats.data._
import cats.implicits._
import cats.effect._
opaque type Version = String
opaque type Path = String
case class ExampleConfig(version: Version, path: Path)
enum ValidationError:
case InvalidVersion
case InvalidPath
// Implementation irrelevant to the example
def validateVersion(version: String): ValidatedNec[ValidationError, Version] = ???
def validatePath(path: String): ValidatedNec[ValidationError, Path] = ???
def validateConfig(version: String, path: String): ValidatedNec[ValidationError, ExampleConfig] =
(validateVersion(version), validatePath(path)).mapN(ExampleConfig.apply)
However, if we now imagine validatePath actually verifies whether the path exists on the filesystem, it becomes an effect like Sync. This makes this case significantly more complex, as we now have:
an additional error channel that we don't want to use, but do need to deal with
nested types, which quickly become a hassle when mixing effectful and regular validations and when there's dependencies between them, requiring some type juggling
// Implementation irrelevant to the example
def pathExists[F[_]: Sync](path: Path): F[Boolean] = ???
def validatePath2[F[_]: Sync](path: String): F[ValidatedNec[ValidationError, Path]] =
// Such a dependency between validations leads to major painpoints,
// as we now have an effect inside a validation, while we would want it the other way around
validatePath(path)
.fold(
e => Sync[F].delay(e.invalid),
path => Sync[F].ifF(pathExists(path))(path.validNec, ValidationError.InvalidPath.invalidNec)
)
def validateConfig2[F[_]: Sync](version: String, path: String): F[ValidatedNec[ValidationError, ExampleConfig]] =
// We now have a distinction between our regular validations, and effectful validations
val versionValid: ValidatedNec[ValidationError, Version] = validateVersion(version)
for {
pathValid: ValidatedNec[ValidationError, Path] <- validatePath2(path)
// We need to deal with any errors in Syncs error channel,
// in addition to our validation error domain,
// as there technically can be additional errors in here that we want to map to validation errors
.handleError(e => ValidationError.InvalidPath.invalidNec)
} yield (versionValid, pathValid).mapN(ExampleConfig.apply)
Is there a better, idiomatic, way of modeling error accumulating effect using Cats Effect?
Related
I have the following case class:
case class Example[T](
obj: Option[T] | T = None,
)
This allows me to construct it like Example(myObject) instead of Example(Some(myObject)).
To work with obj I need to normalise it to Option[T]:
lazy val maybeIn = obj match
case o: Option[T] => o
case o: T => Some(o)
the type test for Option[T] cannot be checked at runtime
I tried with TypeTest but I got also warnings - or the solutions I found look really complicated - see https://stackoverflow.com/a/69608091/2750966
Is there a better way to achieve this pattern in Scala 3?
I don't know about Scala3. But you could simply do this:
case class Example[T](v: Option[T] = None)
object Example {
def apply[T](t: T): Example[T] = Example(Some(t))
}
One could also go for implicit conversion, regarding the specific use case of the OP:
import scala.language.implicitConversions
case class Optable[Out](value: Option[Out])
object Optable {
implicit def fromOpt[T](o: Option[T]): Optable[T] = Optable(o)
implicit def fromValue[T](v: T): Optable[T] = Optable(Some(v))
}
case class SomeOpts(i: Option[Int], s: Option[String])
object SomeOpts {
def apply(i: Optable[Int], s: Optable[String]): SomeOpts = SomeOpts(i.value, s.value)
}
println(SomeOpts(15, Some("foo")))
We have a specialized Option-like type for this purpose: OptArg (in Scala 2 but should be easily portable to 3)
import com.avsystem.commons._
def gimmeLotsOfParams(
intParam: OptArg[Int] = OptArg.Empty,
strParam: OptArg[String] = OptArg.Empty
): Unit = ???
gimmeLotsOfParams(42)
gimmeLotsOfParams(strParam = "foo")
It relies on an implicit conversion so you have to be a little careful with it, i.e. don't use it as a drop-in replacement for Option.
The implementation of OptArg is simple enough that if you don't want external dependencies then you can probably just copy it into your project or some kind of "commons" library.
EDIT: the following answer is incorrect. As of Scala 3.1, flow analysis is only able to check for nullability. More information is available on the Scala book.
I think that the already given answer is probably better suited for the use case you proposed (exposing an API can can take a simple value and normalize it to an Option).
However, the question in the title is still interesting and I think it makes sense to address it.
What you are observing is a consequence of type parameters being erased at runtime, i.e. they only exist during compilation, while matching happens at runtime, once those have been erased.
However, the Scala compiler is able to perform flow analysis for union types. Intuitively I'd say there's probably a way to make it work in pattern matching (as you did), but you can make it work for sure using an if and isInstanceOf (not as clean, I agree):
case class Example[T](
obj: Option[T] | T = None
) {
lazy val maybeIn =
if (obj.isInstanceOf[Option[_]]) {
obj
} else {
Some(obj)
}
}
You can play around with this code here on Scastie.
Here is the announcement from 2019 when flow analysis was added to the compiler.
I'm using Slick on a project and for that I require a Slick representation of my rows and then also an in memory representation. I'm going to use a much simpler example here for brevity. Say for example I have both these types:
type RawType =
(String, Int, Boolean)
type RawTypeRep =
(Rep[String], Rep[Int], Rep[Boolean])
Is there a way to generate one from the other so I don't have to update them in lockstep?
Or perhaps generate them both from a case class? I do have a case class representation, but it's actually slightly different from the types I have because when I hydrate the case class I do some mutations which results in type changes.
This seems like a job for shapeless, though I'm not sure exactly the best way to apply it to your case. The best I can think of is not to generate one from the other, but to at least verify at compile time that the two match up:
import shapeless._
import shapeless.poly._
import shapeless.ops.tuple._
object ToRep extends (Id ~> Rep) {
def apply[A](a: A): Rep[A] = ???
}
type CheckRep[A, B] = Mapper.Aux[A, ToRep.type, B]
type RawType = (String, Int, Boolean)
type RawTypeRep = (Rep[String], Rep[Int], Rep[Boolean])
type RawTypeRepWrong = (Rep[String], Rep[String], Rep[Boolean])
implicitly[CheckRep[RawType, RawTypeRep]] // compiles
implicitly[CheckRep[RawType, RawTypeRepWrong]] // compile error
I am guessing there's probably a better way to handle this given a step back to a larger context. Might be worthwhile to skim the documentation for shapeless to see the kinds of things it can do and see if that gives you any idea.
What would it be the best approach to solve this problem in the most functional (algebraic) way by using Scala and Cats (or maybe another library focused on Category Theory and/or functional programming)?
Resources
Provided we have the following methods which perform REST API calls to retrieve single pieces of information?
type FutureApiCallResult[A] = Future[Either[String, Option[A]]]
def getNameApiCall(id: Int): FutureApiCallResult[String]
def getAgeApiCall(id: Int): FutureApiCallResult[Int]
def getEmailApiCall(id: Int): FutureApiCallResult[String]
As you can see they produce asynchronous results. The Either monad is used to return possible errors during API calls and Option is used to return None whenever the resource is not found by the API (this case is not an error but a possible and desired result type).
Method to implement in a functional way
case class Person(name: String, age: Int, email: String)
def getPerson(id: Int): Future[Option[Person]] = ???
This method should used the three API calls methods defined above to asynchronously compose and return a Person or None if either any of the API calls failed or any of the API calls return None (the whole Person entity cannot be composed)
Requirements
For performance reasons all the API calls must be done in a parallel way
My guess
I think the best option would be to use the Cats Semigroupal Validated but I get lost when trying to deal with Future and so many nested Monads :S
Can anyone tell me how would you implement this (even if changing method signature or main concept) or point me to the right resources? Im quite new to Cats and Algebra in coding but I would like to learn how to handle this kind of situations so that I can use it at work.
The key requirement here is that it has to be done in parallel. It means that the obvious solution using a monad is out, because monadic bind is blocking (it needs the result in case it has to branch on it). So the best option is to use applicative.
I'm not a Scala programmer, so I can't show you the code, but the idea is that an applicative functor can lift functions of multiple arguments (a regular functor lifts functions of single argument using map). Here, you would use something like map3 to lift the three-argument constructor of Person to work on three FutureResults. A search for "applicative future in Scala" returns a few hits. There are also applicative instances for Either and Option and, unlike monads, applicatives can be composed together easily. Hope this helps.
You can make use of the cats.Parallel type class. This enables some really neat combinators with EitherT which when run in parallel will accumulate errors. So the easiest and most concise solution would be this:
type FutureResult[A] = EitherT[Future, NonEmptyList[String], Option[A]]
def getPerson(id: Int): FutureResult[Person] =
(getNameApiCall(id), getAgeApiCall(id), getEmailApiCall(id))
.parMapN((name, age, email) => (name, age, email).mapN(Person))
For more information on Parallel visit the cats documentation.
Edit: Here's another way without the inner Option:
type FutureResult[A] = EitherT[Future, NonEmptyList[String], A]
def getPerson(id: Int): FutureResult[Person] =
(getNameApiCall(id), getAgeApiCall(id), getEmailApiCall(id))
.parMapN(Person)
this is the only solution i came across with but still not satisfied because i have the feeling it could be done in a cleaner way
import cats.data.NonEmptyList
import cats.implicits._
import scala.concurrent.Future
case class Person(name: String, age: Int, email: String)
type FutureResult[A] = Future[Either[NonEmptyList[String], Option[A]]]
def getNameApiCall(id: Int): FutureResult[String] = ???
def getAgeApiCall(id: Int): FutureResult[Int] = ???
def getEmailApiCall(id: Int): FutureResult[String] = ???
def getPerson(id: Int): FutureResult[Person] =
(
getNameApiCall(id).map(_.toValidated),
getAgeApiCall(id).map(_.toValidated),
getEmailApiCall(id).map(_.toValidated)
).tupled // combine three futures
.map {
case (nameV, ageV, emailV) =>
(nameV, ageV, emailV).tupled // combine three Validated
.map(_.tupled) // combine three Options
.map(_.map { case (name, age, email) => Person(name, age, email) }) // wrap final result
}.map(_.toEither)
Personally I prefer to collapse all non-success conditions into the Future's failure. That really simplifies the error handling, like:
val futurePerson = for {
name <- getNameApiCall(id)
age <- getAgeApiCall(id)
email <- getEmailApiCall(id)
} yield Person(name, age, email)
futurePerson.recover {
case e: SomeKindOfError => ???
case e: AnotherKindOfError => ???
}
Note that this won't run the requests in parallel, to do so you'd need to move the future's creation outside of the for comprehension, like:
val futureName = getNameApiCall(id)
val futureAge = getAgeApiCall(id)
val futureEmail = getEmailApiCall(id)
val futurePerson = for {
name <- futureName
age <- futureAge
email <- futureEmail
} yield Person(name, age, email)
If I have multiple operations that return a Validation[E, _] of something with a fixed error type I can use them in a for-comprehension. For example:
val things: Validation[E, (Int, Double)] = for {
i <- getValidationOfInt
d <- getValidationOfDouble
} yield (i, d)
What about if the error types are different? Suppose I read from HTTP and want to convert the string response into an Int.
import scalaz._; import Scalaz._
object ValidationMixing {
class HttpError
def getFromHttp: Validation[HttpError, String] = ???
def parseInt(json: String): Validation[Throwable, Int] =
Validation.fromTryCatchNonFatal(Integer.parseInt(json))
val intParsedFromHttp: Validation[Any, Int] = for {
s <- getFromHttp
i <- parseInt(s)
} yield i
}
This compiles, but only because the error type of the Validation is Any, being a supertype of Throwable and HttpError. This isn't very helpful.
I can think of various ways of representing such a combined error type that are more useful than Any (e.g. Validation[Error1 \/ Error2, Result] to store either, Validation[String, Result] translating to an error message, etc) but they all have drawbacks.
Is there an idiomatic way to do this?
Since nobody has had a better idea I'll leave my answer for future reference.
As said in the comment the best way is to create a error hierarchy:
trait GenericError { /* some commond fields */}
case class MyNumericError(/* fields */)
and then use leftMap on a validation to generate appropriate errors:
Validation.fromTryCatchNonFatal(...).leftMap(t => MyNumericError(...))
This approach has two advantages
First you will always have a Validation[GenericError, T] so not having different types on the left part of that validation
The second is that this helps generate meaningful errors for both developers and service user, also note that generating an error where you have many context informations helps in this process.
We are building some sync functionality using two-way json requests and this algorithm. All good and we have it running in prototype mode. Now I am trying to genericise the code, as we will be synching for several tables in the app. It would be cool to be able to define a class as "extends Synchable" and get the additional attributes and sync processing methods with a few specialisations/overrides. I have got this far:
abstract class Synchable [T<:Synchable[T]] (val ruid: String, val lastSyncTime: String, val isDeleted:Int) {
def contentEquals(Target: T): Boolean
def updateWith(target: T)
def insert
def selectSince(clientLastSyncTime: String): List[T]
def findByRuid(ruid: String): Option[T]
implicit val validator: Reads[T]
def process(clientLastSyncTime: String, updateRowList: List[JsObject]) = {
for (syncRow <- updateRowList) {
val validatedSyncRow = syncRow.validate[Synchable]
validatedSyncRow.fold(
valid = { result => // valid row
findByRuid(result.ruid) match { //- do we know about it?
case Some(knownRow) => knownRow.updateWith(result)
case None => result.insert
}
}... invalid, etc
I am new to Scala and know I am probably missing things - WIP!
Any pointers or suggestions on this approach would be much appreciated.
Some quick ones:
Those _ parameters you pass in and then immediately assign to vals: why not do it in one hit? e.g.
abstract class Synchable( val ruid: String = "", val lastSyncTime: String = "", val isDeleted: Int = 0) {
which saves you a line and is clearer in intent as well I think.
I'm not sure about your defaulting of Strings to "" - unless there's a good reason (and there often is), I think using something like ruid:Option[String] = None is more explicit and lets you do all sorts of nice monad-y things like fold, map, flatMap etc.
Looking pretty cool otherwise - the only other thing you might want to do is strengthen the typing with a bit of this.type magic so you'll prevent incorrect usage at compile-time. With your current abstract class, nothing prevents me from doing:
class SynchableCat extends Synchable { ... }
class SynchableDog extends Synchable { ... }
val cat = new SynchableCat
val dog = new SynchableDog
cat.updateWith(dog) // This won't end well
But if you just change your abstract method signatures to things like this:
def updateWith(target: this.type)
Then the change ripples down through the subclasses, narrowing down the types, and the compiler will omit a (relatively clear) error if I try the above update operation.