Service class and logging design Scala with Futures - scala

I have a service class that gets some data from a database (for context, I'm using Play! Framework). Here's an example method:
def getAccessToken(id: BSONObjectID): Future[Option[String]] = {
userDAO.find(id).map {
case Some(user) =>
user.settings flatMap (_.accessToken)
case _ => None
}
}
I'm trying to improve the error handling (new to Scala) on this thing as there are a couple things that can go wrong:
User might not be found
User might be found but accessToken might not be set (accessToken is an Option[String])
As it stands right now, I can't differentiate between the two. My natural inclination to fix this is to use \/ from Scalaz and have the return type be Future[ErrorType \/ String] and this seems like a reasonable approach. In my controller method, I can for comprehension a bunch of different service methods like this by lifting into a wrapper monad.
But I have the following questions:
Should my ErrorType extend Exception, or should I just use the sealed trait style and just extend from that. I've heard that it is not good practice to use exceptions in Scala, so I'm not sure what the right approach is.
How can I handle logging without polluting the controller class with excessive log statements? If a controller class calls a bunch of these service methods, the controller will have to handle several different ErrorTypes in the for comprehension. Assuming I'm lifting all the monads to a wrapper monad with ?|, I want to avoid this:
accessToken <- service.getAccessToken(id) ?| { error => error match { case Error1 =>
logger.error(
"Unable to find access token for user: " + id
.toString())
InternalServerError(
ApiResponse("internal_server_error",
"Unable to get token."))
case Error2 => ...
}
Thanks!

I think Future[ErrorType / String] is a bit overkill as Future[T] already can hold either an object of type T or an Exception derived one (see Future.successful(...)/ Future.failed(...))
Should my ErrorType extend Exception, or should I just use the sealed trait style and just extend from that. I've heard that it is not good practice to use exceptions in Scala, so I'm not sure what the right approach is.
I would recommend to use a class (or a set of classes, one per particular error type), say YourAppException derived from Exception, as you need to handle low level exceptions one way or another anyway.
I agree that throwing/catching exceptions is not going very well with functional code and it is better to use Try[T] or Future[T] to return errors in a more explicit way instead. On the other hand there is nothing wrong with using an Exception derived class to hold some error info. It is often useful to wrap the original non-application (say IO) exception in an application one and keep a reference to the initial one in Exception's 'cause' for troubleshooting. It gives an opportunity to provide a more context specific error message.
How can I handle logging without polluting the controller class with excessive log statements?
Consider encapsulating error messages in Exception derived case classes, representing application errors so you can access the error messsages uniformly using exception.getMessage. It is easy to add some method to YourAppException to construct ApiResponse as well.
def getAccessToken(id: BSONObjectID): Future[String] = {
userDAO.find(id).flatMap {
case Some(user) =>
val optToken = user.settings.flatMap (_.accessToken)
optToken.map(Future.successful).getOrElse(Future.failed(AccessTokenIsInvalid(user)))
case _ => Future.failed(UserNotFoundError(user))
}
}
case class AccessTokenIsInvalid(user: String)
extends YourAppException(s"Access token is invalid for user $user") {
}
accessToken <- service.getAccessToken(id) ?| { error =>
logger.error(error.getMessage)
InternalServerError(
ApiResponse("internal_server_error", error.getMessage))
}

1) Yes, you are on a right way. The problem with exceptions is that when something fails it's hard to pattern-match on it to detect a reason.
I would do it like that:
sealed trait MyError
object UserNotFound extends MyError
object AuthFailed extends MyError
type MyResult = Either[MyError, String]
2) If a program is well-typed logging is necessary in places where information is lost.
If you deal with, for example, val x = Future[Either[Error, String]], then you haven't throttled potential error yet so logging is optional.
But when you somehow try to extract Either[MyError, String] from it you lose information so you should log it.
The same happens when you extract String from Either[MyError, String].

Related

Using Try out of Exception contexts

Is it in Scala acceptable to use Try (Success/Failure) outside of an actual Exception context as a return type of a function that can succeed or fail?
It is certainly possible to use a Try outside an exception context; I use it that way all the time. That does not necessarily mean that it is "acceptable" :)
I would say that the whole point of using Try is to take the Throwable instance out of the exception context and put it in an object that can be used anywhere in a program. It is hard to see why Try would have such a rich set of methods (e.g. flatMap) if it is only intended to be used inside an exception context.
Instances of Try, are either Success or Failure, where Failure is
case class Failure[+T](exception: Throwable)
Note how Failure must be constructed with Throwable, so I think Try is meant to be used within context of Throwables. So we cannot do something like
def foo: Try[Int] = {
Failure(42) // Error: type mismatch; found : Int(42) required: Throwable
}
Consider using Either instead of Try outside exceptions context.
Addressing the comment consider
Valid/Invalid from cats: https://typelevel.org/cats/datatypes/validated.html
define your own ADT with your own meaning of success and failure cases, and then wrap function results in those cases
In neither of these are you forced to use exceptions.
Here is an example
sealed trait MyValidationADT[T]
case class Good[T](result: T) extends MyValidationADT[T]
case class Bad[T](result: T) extends MyValidationADT[T]
def foo(i: Int): MyValidationADT[Int] = Bad(42)
foo(11) match {
case Good(result) => "woohoo"
case Bad(result) => "boom"
}
which outputs
res0: String = boom

Use Future.successful(None) vs Future.failed(new Exception()) for an API that may return no results?

What is the idiomatic way in Scala to type an API that may or may not return results?
I can simply type to Future[Option[A]] and return a None if there is no results, but this will make things a little more verbose for API clients I think because it has to do some pattern matching (or other techniques like folding) over the Option returned.
trait SomeAPI {
def fetch(): Future[Option[String]]
}
object APIImpl {
def asyncDBCall(): Future[List[String]] = ???
override def fetch(): Future[Option[String]] = asyncDBCall().map(r => if (r.isEmpty) None else Some(r.head)
}
However, Futures already give us a way to say that there are no results making it fail. But I don't know if this is a proper or expressive way of say that the call doesn't return results.
trait SomeAPI {
def fetch(): Future[String]
}
class NoResultsException extends Exception
object APIImpl extends SomeAPI {
def asyncDBCall(): Future[List[String]] = ???
override def fetch(): Future[String] = asyncDBCall().map(r => if (r.isEmpty) throw new NoResultsException() else r.head)
}
At first it seems that both options are acceptable and the final decision may be just about personal preference, but maybe I'm missing something and make a Future fail is not a good option.
Exceptions should be reserved for errors. If the API may or may not returns results, and both cases are considered normal, then the API should use Option.
If that makes it more verbose by forcing the client to handle both situations, that's good, because the client should handle both situations. Exceptions are good when you don't want to force the client code to handle an exceptional case (unexpected error) which should probably cascade and be handled by an error handler.
Now, if lack of results truly indicates that something went wrong, then an exception would be appropriate.
I'd expect clients want to treat "success, no results" and failure differently, generally speaking (e.g. retry in the second case, but not the first). If you denote the first with a failed future, you make verbosity for the clients worse, not better: they just need to check the exception instead of the success, and the API isn't helping them to be aware of that like Future[Option[A]] would.
If you really worry about verbosity, you could add custom extractors or methods for the clients to use if they want, e.g.
implicit class FutureOption[A](future: Future[Option[A]]) {
def mapOpt[B](handleSuccessWithResult: A => B, handleSuccessWithNoResult: () => B) = Future.map {
case Some(x) => onSuccessWithResult(x)
case None => onSuccessWithNoResult()
}
}
The others are right. Failures are not a good way to describe the lack of a concrete result, if this is part of regular operation.
Scala gives you another possibility to explicitly note, that there may be no results, sealed traits.
sealed trait MyOperationReturn
case object OkNoResult
case class OkWithStringResponse(data: String) extends MyOperationReturn
case class OkWithIntResponse(data: Int) extends MyOperationReturn
The user can then just pattern match. This option is especially handy, if you have either multiple return types (not only two) and no natural empty return type (like Seq.empty)

Scala: line by line check for None, or check for exceptions at the end?

I have a case class Application that has some input ports, and each port has a name. Then I have another case class that assigns values to the ports of an application.
case class Port (id: ObjectId, name: String, PortType: String)
case class Application (id: ObjectId, ports: List[Port])
case class AppRun (appId: ObjectId, assignments: List[Assignment])
case class Assignment (portName: String, value: String, valueType: String)
I have the applications and their port information in a database, and I get as input an AppRun. I need to make a list of PortValue type (below) showing the value assigned to each port (and the matching is done on port names):
case class PortValue (portId: ObjectId, value: String)
There are a few things that may fail during this matching: application id is invalid, ports do not match, etc. It feels natural to me to write the straightforward algorithm and then catch all exceptions, but that seems Java-ish. On the other hand, I cannot think of a neat way of dealing with Options , checking them one by one, which will obfuscate the code.
The question is how would you solve this Scala way?
EDIT: I need to send a proper message back when such a mismatch happens, like "application not found", etc.
A way to deal with checking Options one by one is to use a for-comprehension. And if you want to keep track of errors you can quite often replace Option with some class that does error-tracking. The common possibilities include:
scala.util.Try[T]. Try is either a Success(result), or a Failure(error: Throwable). It is a built-in class in Scala, and it is simple to combine it with or replace it by scala.concurrent.Future if the need arises.
scala.util.Either[E, T]. Creating a Throwable for every error may not be very efficient because of the need to build the stacktrace. So Either is useful if the error can be a simple String or some application-specific class without the stacktrace. The convention is to have a Right(result) or a Left(error). The downsides are that it's not semantic to have 'right' mean 'success' and 'left' mean 'error', and when you use it in a for-comprehension or call e.g. map method on it, you have to specify whether you want either.right or either.left.
scalaz.\/[E, T] This is the same as Either, but the default for map and for-comprehension is its right side (\/-). Also scalaz provides very useful functions sequence and traverse (see the code below).
scalaz.Validation[Errors, T] or scalaz.ValidationNel[E, T]. Adds a very useful functionality of collecting all the errors, but has slight problems when used in for-comprehensions.
Here is some sample code for your problem, using Try:
import scala.util.{Try, Success, Failure}
def getApplication(appId: ObjectId): Option[Application] = ???
/** Convert Option to Try, using a given failure in case of None */
def toTry[T](option: Option[T])(failure: => Throwable): Try[T] =
option.fold[Try[T]](Failure(failure))(Success(_))
/** Convert a List of Try to a Try of List.
* If all tries in the List are Success, the result is Success.
* Otherwise the result is the first Failure from the list */
def sequence[T](tries: List[Try[T]]): Try[List[T]] =
tries.find(_.isFailure) match {
case Some(Failure(error)) => Failure(error)
case _ => Success(tries.map(_.get))
}
def traverse[T, R](list: List[T])(f: T => Try[R]): Try[List[R]] =
sequence(list map f)
def portValues(task: AppRun): Try[List[PortValue]] = for {
app <- toTry(getApplication(task.appId))(
new RuntimeException("application not found"))
portByName = app.ports.map(p => p.name -> p).toMap
ports <- traverse(task.assignments) { assignment =>
val tryPort = toTry(portByName.get(assignment.portName))(
new RuntimeException(s"no port named ${assignment.portName}"))
tryPort.map(port => PortValue(port.id, assignment.value))
}
} yield ports
Some considerations:
Provided implementations of toTry, sequence and traverse are just a sample. For one, I'd define them in implicit classes to be able to call them like normal methods (e.g. option.toTry(error), or list.traverse(f)).
traverse can be implemented more effectively (stop after the first error is found).
this sequence implementation would return only the first erroneous port.
I prefer API like def getApplication(id: ObjectId): Try[Application] instead of an Option result, because you usually want to have the same error in every part of the code that calls it, and it may give different errors as well (e.g., id not found or network error). If you have def getApplication(id: ObjectId): Application that may throw an error you can simply wrap it in Try: for { app <- Try(getApplication(id)) ...

Making code more scala idiomatic

I came across following java like code in Scala project. how to make it more Scala idiomatic with no side effects (exception handling appropriately) ?
I am thinking to use scalaz disjunction / (I know I can use Scala either too but like right biased more I guess). in a function there are a few such if checks(one above is one example) which throw one or the other type of exceptions. how to make such code more Scala idiomatic?
EDIT:
Question is not around how to convert Java null checks into Scala Idiomatic, which I am already doing. for e.g. following
hpi.fold(throw new Exception("Instance not found for id " + processInstanceId)) { h =>
val pi = new ProcessInstance(taskResponse)
Now return type of the existing function is some value say "ProcessInstance" for e.g. but in my opinion is misleading. caller would never know if this will throw an exception so my question is more around returning Either[Error,Value] from such functions. And if I have a few such exceptions being captured into a single function, how to accumulate them all and reflect into return type?
One thought might making processDefinition.getDiagramResourceName() return an Option, so you can then check whether the result is Some(x) or None.
Using scalaz and being idiomatic this is probably what I would end up with (subject to refactoring):
for {
pd <- Option(processDefinition.getDiagramResourceName()).\/>("Diagram resource could not be found")
} yield pd
So if you have null you get back Left("Diagram resource could not be found") otherwise you get Right(DiagramResourceName).
An approach based in case classes where different subclasses of names, references or labels may be defined,
trait ResourceName
case object MissingName extends ResourceName
case class DiagramResourceName(name: String) extends ResourceName
case class AnotherResourceName(ref: Int) extends ResourceName
processDefinition.getDiagramResourceName() match {
case DiagramResourceName(name) => println(s"name $name")
case MissingName => throw new ActivityException(errorMessage)
}

Get Option value or throw an exception

Given an Option, what is the idiomatic way to get its value or throw an exception trying?
def foo() : String = {
val x : Option[String] = ...
x.getOrException()
}
A throw "statement" is really an expression in Scala, and it has type Nothing, which is a subtype of every other type. This means you can just use plain old getOrElse:
def myGet[A](oa: Option[A]) = oa.getOrElse(throw new RuntimeException("Can't."))
You really, really shouldn't be doing this, though.
(EDIT: this is not the best or most idiomatic way to do it. I wrote it when I was not familiar with Scala. I leave it here for an example of how not to do it. Nowadays I would do as #TravisBrown)
I think it really boils down to two things:
how sure are you that the value is there?
how do you want to react if it isn't?
If at that point in your code you expect the value to be there, and in the remote case that it isn't you want your program to fail fast, then I would only do a normal get and let Scala throw a NoSuchElementException if there was no value:
def foo() : String = {
val x : Option[String] = ...
x.get
}
If you want to handle the case differently (throw your own exception) I think a more elegant way would look like this:
def foo(): String = {
val x: Option[String] = None
x match {
case Some(value) => value
case None => throw new MyRuntimeException("blah")
}
}
And of course if you want to supply your own alternative value for the case that the Option is None you would just use getOrElse:
def foo(): String = {
val x: Option[String] = None
x.getOrElse("my alternative value")
}
I hope this will help you to understand how to represent errors (and generally effects) using types.
Error handling strategies in functional Scala
Use Option to return optional values. For example - fail to find entity in storage.
Use Option(possiblyNull) to avoid instances of Some(null).
Use Either[Error, T] to report expected failure. For example - email format is wrong, cannot parse a string to a number, etc.
Model your errors as ADTs (simply speaking kind of type hierarchies) to use it, for example, on the Left of the Either to represent more complex error scenarios.
Throw Exception only to signal unexpected and not-recoverable failures. Like missing config file.
Use Either.catchOnly or Try or Cats.IO (advanced) rather than a catch block for handling unexpected failures. Hint: You can still use ADT but extend them from throwables. More about Either vs Try.
Use Validated data-type from Cats lib to accumulate errors rather than fail-fast (Either), but prefer Either's on module-level to simplify the composition of the program (to have the same types). For example - form data validation, parsing errors accumulation.
Use mentioned types and don't optimize program preemptively - since most probably, bottle-necks would be in business logic, not in effect types.
Such an approach will simplify maintenance and updates of your code since you can reason about it without going to implementation specifics (aka local-reasoning). Also - reduce bugs - you cannot miss an error in the type. And compose the program easier (with help of map, flatMap and other combinators) - since it's simpler on type level, rather than with non-local exceptions and side-effects.
More about learning functional Scala.
But be aware that sometimes with this approach types could stack up and it could become harder to compose things. Given, for example: x: Future[Either[Error, Option[T]]] What you can do:
Use map and flatMap in combination with pattern-matching to compose different values of such types, for example:
x.faltMap { case Right(Some(v)) => anotherFuture(v); case Left(er) => ... }
If it doesn't help you can try to use MonadTransformers (don't be scared of the name, it's just wrappers around the effect types like Either and Future)
Also, an option is to simplify your errors ADT by extending them from the Throwable to unify it with Future, then it'll be Future[Option[T]]
And finally, in your case one option will be:
def foo() : Either[Error, String] = {
val x : Option[String] = ...
x match {
case Some(v) => Right(v)
case None => Left(Error(reason))
}
}
Just use the .get method.
def get[T](o:Option[T]) = o.get
It will throw a NoSuchElementException if o is an instance of None.
Basically, I would work with options like this:
def addPrint(oi:Option[Int]) = oi.map(_+1).foreach(println)
addPrint(Some(41))
addPrint(Some(1336))
addPrint(None)
to avoid your specific question.
Scala now support this operation on maps using getOrElse() method, see documentation here
As pointed out already, throwing an exception in Scala is an expression as well.
So you can do the following:
myMap.getOrElse(myKey, throw new MyCustomException("Custom Message HERE")