In regard to potential runtime failures, like database queries, it seems that one must use some form of Either[String, Option[T]] in order to accurately capture the following outcomes:
Some (record(s) found)
None (no record(s) found)
SQL Exception
Option simply does not have enough options.
I guess I need to dive into scalaz, but for now it's straight Either, unless I'm missing something in the above.
Have boxed myself into a corner with my DAO implementation, only employing Either for write operations, but am now seeing that some Either writes depend on Option reads (e.g. checking if email exists on new user signup), which is a majorly bad gamble to make.
Before I go all-in on Either, does anyone have alternate solutions for how to handle the runtime trifecta of success/fail/exception?
Try Box from the fantastic lift framework. It provides exactly what you want.
See this wiki (and the links at the top) for details. Fortunately lift project is well modulized, the only dependency to use Box is net.lift-web % lift-common
Use Option[T] for the cases records found and no records found and throw an exception in the case of SQLException.
Just wrap the exception inside your own exception type, like PersistenceException so that you don't have a leaky abstraction.
We do it like this because we can't and don't want to recover from unexpected database exceptions. The exception gets caught on the top level and our web service returns a 500 Internal server error in such case.
In cases where we want to recover we use Validation from scalaz, which is much like Lift's Box.
Here's my revised approach
Preserve Either returning query write operations (useful for transactional blocks where we want to rollback on for comprehension Left outcome).
For Option returning query reads, however, rather than swallowing the exception with None (and logging it), I have created a 500 error screen, letting the exception bubble up.
Why not just work with Either result type by default when working with runtime failures like query Exceptions? Option[T] reads are a bit more convenient to work with vs Either[Why-Fail, Option[T]], which you have to fold/map through to get at T. Leaving Either to write operations simplifies things (all the more so given that's how the application is currently setup, no refactoring required ;-))
The only other change required is for AJAX requests. Rather than displaying the entire 500 error page response in the AJAX status div container, we check for the status type and display 500 error message accordingly.
if(data.status == 500)
$('#status > div').html("an error occurred, please try again")
Could probably do an isAjax check server-side prior to sending the response; in which case I can send back only status + message rather than the error page itself.
Related
ZIO (https://zio.dev/) is a scala framework which has at its core the ZIO[R, E, A] datastructure and its site gives the following information for the three parameters:
ZIO
The ZIO[R, E, A] data type has three type parameters:
R - Environment Type. The effect requires an environment of type R. If this type parameter is Any, it means the effect has no
requirements, because you can run the effect with any value (for
example, the unit value ()).
E - Failure Type. The effect may fail with a value of type E. Some applications will use Throwable. If this type parameter is
Nothing, it means the effect cannot fail, because there are no
values of type Nothing.
A - Success Type. The effect may succeed with a value of type A. If this type parameter is Unit, it means the effect produces no
useful information, while if it is Nothing, it means the effect runs
forever (or until failure).
It's easy to get what A is: it's the value returned by the function in the nominal case, ie why we coded the function for.
R is so kind of dependency injection - an interesting topic, but we can just ignore it to use ZIO by alway setting it to Any (and there is actually a IO[E, A] = ZIO[Any, E, A] alias in the lib).
So, it remains the E type, which is for error (the famous error channel). I roughtly get that IO[E, A] is kind of Either[E, A], but deals with effect (which is great).
My question is: why should I use an error channel EVERYWHERE in my application, and how can I decide what should go in the error channel?
1/ Why effect management with an error channel?
As a developper, one of your hardest task is to decide what is an error and what is not in your application - or more preciselly, to discover failure modes: what the nominal path (ie the goal of that code), what is an expected error that can be dealt with by the application in some way later on, and what are unexpected errors that the application can't deal with. There is no definitive answer for that question, it depends of the application and context, and so it's you, the developper, who needs to decide.
But the hardest task is to build an application that keeps its promises (your promises, since you chose what is an error and what is the nominal path) and that is not surprising so that users, administrators, and dev - including the futur you in two weeks - know what the code do in most cases without having to guess and have agency to adapt to that behavior, including to respond to errors.
This is hard, and you need a systematic process to deals with all the possible cases without going made.
The error channel in IO bi-monad (and thus ZIO) helps you for that task: the IO monad helps you keep track of effects, which are the source of most errors, and the error channel makes explicit what are the possible error cases, and so other parts of the application have agency to deal with them if they can. You will be able to manage your effects in a pure, consistant, composable way with explicit failure modes.
Moreover, in the case of ZIO, you can easely import non-pure code like legacy java extremelly easily:
val pure = ZIO.effect(someJavaCodeThrowingException)
2/ How do I choose what is an error?
So, the error channel provide a way to encode answer to what if? question to futur dev working on that code. "What if database is down?" "there's a DatabaseConnectionError".
But all what if are not alike for YOUR use case, for CURRENT application level. "What if user is not found?" - ah, it may be a totally expected answer at the low, "repository" level (like a "find" that didn't find anything), or it can be an error at an other level (like when you are in the process of authenticating an user, it should really be there). On the first case, you will likely not use the error channel: it's the nominal path, sometimes you don't find things. And in the second case, you will likelly use the error channel (UserNotFoundError).
So as we said, errors in error channel are typically for what if question that you may want to deal with in the application, just not at that function level. The first example of DatabaseConnectionError may be catch higher in the app and lead to an user message like "please try again" and a notification email to sysadmin ("quick, get a look, something if wrong here"). The UserNotFoundError will likely be managed as an error message for the user in the login form, something like "bad login or password, try again or recover credentials with that process".
So these cases (nominal and expected errors) are the easy parts. But there are some what if questions that your application, whatever the level, has no clue how to answer. "What if I get a memory exception when I try to allocate that object?" I don't have any clue, and actually, even if I had a clue, that's out of the scope of the things that I want to deal with for that application. So these errors DON'T go in the error channel. We call them failure and we crash the application when they happens, because it's likely that the application is now in an unknow, dangerous, zombie state.
Again, that choice (nominal path/error channel/failure) is your choice: two applications can make different choices. For example, a one-time-data-processing-app-then-discard-it will likelly treat all non-nominal paths as failures. There is a dev to catch the case in realtime and decide if it's important (see: Shell, Python, and any scripting where that strategy is heavely used - ok, sometimes even when there is no dev to catch errors:). On the other end of the specter, Nasa dev put EVERYTHING in the error channel(+), even memory CORRUPTION. Because it is an expected error, so the application need to know how to deal with that and continue.
(+)NOTE: AFAIK they don't use zio (for now), but the decision process about what is an error is the same, even in C.
To go further, I (#fanf42) gave a talk at Scala.io conference. The talk, "Ssytematic error management in application", is available in French here. Yes, French, I know - but slides are available in English here! And you can ping me (see contact info near the end of slide deck).
I was accessing REST calls, when I passed wrong parameter to GET request it does not throw any http error. Should the design be changed to throw a http error or wrong parameter can be passed to REST call.
Example 1:(parameters are optional)
https://example.com/api/fruits?fruit=apple
Give list of all apple elements
Example 2:
https://example.com/api/fruits?abc=asb
Give list of all fruits
My question is related to example 2, should example 2 throw an error or is it behaving properly?
It's pretty common to ignore parameters that you aren't necessarily expecting. I think example 2 is behaving as it should.
I know that depending on the browser I would sometimes append an extra variable with a timestamp to make sure that the rest call wouldn't be cached. Something like:
https://example.com/api/fruits?ihateie=2342342342
If you're not explicitly doing anything with the extra parameter then I can't see the harm in allowing it.
For a GET request, the request-line is defined as follows
request-line = 'GET' SP request-target SP HTTP-version CRLF
where request-target "...identifies the target resource upon which to apply the request".
That means that the path /api/fruits, the question-mark ? and the query abc=asb are all part of the identifier.
The fact that your implementation happens to use the path to route the request to a handler, and the query to provide arguments, is an accident of your current implementation.
That leaves you with the freedom to decide that
/api/fruits?abc=asb does exist, and its current state is a list of all fruits
/api/fruits?abc=asb does exist, and its current state is an empty list
/api/fruits?abc=asb does exist, and its current state is something else
/api/fruits?abc=asb does not exist, and attempting to access its current state is an error.
My question is related to example 2, should example 2 throw an error or is it behaving properly?
If abc=asb indicates that there is some sort of error in the client, then you should return a 4xx status to indicate that.
Another way of thinking about the parameter handling is in terms of Must Ignore vs Must Understand.
As a practical matter, if I'm a consumer expecting that my filter is going to result in a small result set, and instead I end up drinking a billion unfiltered records out of a fire hose, I'm not going to be happy.
I'd recommend that in the case of a bad input you find a way to fail safely. On the web, that would probably mean a 404, with an HTML representation explaining the problem, enumerating recognized filters, maybe including a web form that helps resend the query, etc. Translate that into your API in whatever way makes sense.
But choosing to treat that as a successful request and return some representation also works, it's still REST, the web is going to web. If doing it that way gives you consumers a better experience, thereby increasing adoption and making your api more successful, then the answer is easy.
I was looking at scala.util.control.NonFatal. I can't find the source, but I believe it is something like this.
They are declaring LinkageError as Fatal ...
Tomcat (at least last few years I used it) always returned 500 on catch Throwable, rather than crashing on certain kinds of errors. So do many other systems that make a best effort to always return something to the client.
So, my end question is when would you use NonFatal instead of making a best-effort attempt to provide some response?
As an example, now Futures in Twitter's Future library end up not resolving on NoSuchMethodError so my Future no longer resolves as failed with a Throwable but instead throw up the stack (differently from RuntimeException). In fact, in the open source Finagle stack, a NoSuchMethodError will cause the client socket connection to close on the client with no 500 http error back to customer. Customer then thinks 'hmm, network issue maybe ... why did my socket close'
So far, it has caused me nothing but issues and I admit to be a little frustrated, but need to be open to more use cases. For years, KISS and treating every Throwable in the catchall as non fatal has worked, but NonFatal is implying there are use-cases where we should do something different.
The source code of NonFatal is linked from the API docs.
Fatal errors are those from which your system or the JVM will most likely not recover correctly, so catching those errors is not a good idea.
The sub-classes of LinkageError are: ClassCircularityError, ClassFormatError, ExceptionInInitializerError, IncompatibleClassChangeError, NoClassDefFoundError, UnsatisfiedLinkError, VerifyError. These all occur when your class path is broken, there are invalid or binary incompatible class files. It's safe to assume that your entire system is broken if these happen at runtime.
To answer the question: You should "let it crash". Always use a NonFatal pattern match when you need a catch-all clause. It will also do you the favour and handle control-flow related exceptions correctly (e.g. NonLocalReturnControl).
Note that unlike the old source you link to, StackOverflowError is not non-fatal any longer, the decision was revised in Scala 2.11 as per SI-7999.
Beginning in Scala and reading about Either I naturally comparing new concepts to something I know (in this case from Java). Are there any differences from the concept of checked exceptions and Either?
In both cases
the possibility of failure is explicitly annotated in the method (throws or returning Either)
the programmer can handle the error case directly when it occurs or move it up (returning again an Either)
there is a way to inform the caller about the reason of the error
I suppose one uses for-comprehensions on Either to write code as there would be no error similar to checked exceptions.
I wonder if I am the only beginner who has problems to see the difference.
Thanks
Either can be used for more than just exceptions. For example, if you were to have a user either type input for you or specify a file containing that input, you could represent that as Either[String, File].
Either is very often used for exception handling. The main difference between Either and checked exceptions is that control flow with Either is always explicit. The compiler really won't let you forget that you are dealing with an Either; it won't collect Eithers from multiple places without you being aware of it, everything that is returned must be an Either, etc.. Because of this, you use Either not when maybe something extraordinary will go wrong, but as a normal part of controlling program execution. Also, Either does not capture a stack trace, making it much more efficient than a typical exception.
One other difference is that exceptions can be used for control flow. Need to jump out of three nested loops? No problem--throw an exception (without a stack trace) and catch it on the outside. Need to jump out of five nested method calls? No problem! Either doesn't supply anything like this.
That said, as you've pointed out there are a number of similarities. You can pass back information (though Either makes that trivial, while checked exceptions make you write your own class to store any extra information you want); you can pass the Either on or you can fold it into something else, etc..
So, in summary: although you can accomplish the same things with Either and checked exceptions with regards to explicit error handling, they are relatively different in practice. In particular, Either makes creating and passing back different states really easy, while checked exceptions are good at bypassing all your normal control flow to get back, hopefully, to somewhere that an extraordinary condition can be sensibly dealt with.
Either is equivalent to a checked exception in terms of the return signature forming an exclusive disjunction. The result can be a thrown exception X or an A. However, throwing an exception isn't equivalent to returning one – the first is not referentially transparent.
Where Scala's Either is not (as of 2.9) equivalent is that a return type is positively biased, and requires effort to extract/deconstruct the Exception, Either is unbiased; you need to explicitly ask for the left or right value. This is a topic of some discussion, and in practice a bit of pain – consider the following three calls to Either producing methods
for {
a <- eitherA("input").right
b <- eitherB(a).right
c <- eitherC(b).right
} yield c // Either[Exception, C]
you need to manually thread through the RHS. This may not seem that onerous, but in practice is a pain and somewhat surprising to new-comers.
Yes, Either is a way to embed exceptions in a language; where a set of operations that can fail can throw an error value to some non-local site.
In addition to the practical issues Rex mentioned, there's some extra things you get from the simple semantics of an Either:
Either forms a monad; so you can use monadic operations over sets of expressions that evaluate to Either. E.g. for short circuiting evaluation without having to test the result
Either is in the type -- so the type checker alone is sufficient to track incorrect handling of the value
Once you have the ability to return either an error message (Left s) or a successful value Right v, you can layer exceptions on top, as just Either plus an error handler, as is done for MonadError in Haskell.
Many of the Caché API methods return a %Status object which indicates if this is an error. The thing is, when it's an unknown error I don't know how to handle (like a network failure) what I really want to do is "throw" the error so my code stops what it's doing and the error gets caught by some higher level error handler (and/or the built-in %ETN error log).
I could use ztrap like:
s status = someObject.someMethod()
ztrap:$$$ISERR(status)
But that doesn't report much detail (unlike, say, .NET where I can throw an exception all the way to to the top of the stack) and I'm wondering if there are any better ways to do this.
Take a look at the Class Reference for %Exception.StatusException. You can create an exception from your status and throw it to whatever error trap is active at the time (so the flow of control would be the same as your ZTRAP example), like this
set sc = someobj.MethodReturningStatus()
if $$$ISERR(sc) {
set exception = ##class(%Exception.StatusException).CreateFromStatus(sc)
throw exception
}
However, in order to recover the exception information inside the error trap code that catches this exception, the error trap must have been established with try/catch. The older error handlers, $ztrap and $etrap, do not provide you with the exception object and you will only see that you have a <NOCATCH> error as the $ZERROR value. Even in that case, the flow of control will work as you want it to, but without try/catch, you would be no better off than you are with ZTRAP
These are two different error mechanisms and can't be combined in this way. ztrap and %ETN are for Cache level errors (the angle bracket errors like <UNDEFINED>). %Status objects are for application level errors (including errors that occurred through the use of the Cache Class Library) and you can choose how you want to handle them yourself. It's not really meaningful to handle a bad %Status through the Cache error mechanism because no Cache error has occurred.
Generally what most people do is something akin to:
d:$$$ISERR(status) $$$SomeMacroRelevantToMyAppThatWillHandleThisStatus(status)
It is possible to create your own domain with your own whole host of %Status codes with attendant %msg values for your application. Your app might have tried to connect to an FTP server and had a bad password, but that doesn't throw a <DISCONNECT> and there is no reason to investigate the stack, just an application level error that needs to be handled, possibly by asking the user to enter a new password.
It might seem odd that there are these two parallel error mechanisms, but they are describing two different types of errors. Think of one of them being "platform" level errors, and the other as "application level errors"
Edit: One thing I forgot, try DecomposeStatus^%apiOBJ(status) or ##class(%Status).LogicalToOdbc(status) to convert the status object to a human readable string. Also, if you're doing command line debugging or just want to print the readable form to the principal device, you can use $system.OBJ.DisplayError(status).