Unable to properly handle errors - scala

I have found the reason why my application isn't behaving the way I would like it to but I don't know how to resolve the issue. To summarise, my application has a custom error handler which gets invoked if there is an error. The error handler sends json messages. But in one application startup error scenario (Future fails), I want to Redirect the user to homepage instead of sending a json message.but it doesn't happen because the custom error handler sends json message before I can send a Redirect from Future's recover.
One feature of the application is signup verification. The user clicks on a url which has a token. When the url is clicked, verifyUser Action is called. It does some checks (using database queries using Futures) and depending on the success or failure, it sends Redirect with signup=success or signup=error attribute (not failure here is decided based on whether something exists in the database or not). However, if the Future fails (I queried a wrong field which is not part of database's schema), I want to Redirect again but it doesn't work as the custom error handler gets invoked before recover. How could I make my application Redirect?
val result:Future[Result] = for{tokenOption:Option[UserToken] <- userTokenRepo.findOne(UserTokenKey(UUID.fromString(token))) //generator 1 - get token from database
userOption:Option[User] <- if (tokenOption.isDefined) userRepo.findOne(tokenOption.get.userKeys) else Future.successful(None) //generator2. found token, look for corresponding user to which the token belongs
modifiedUser:Option[User] <- if (userOption.isDefined) confirmSignupforUser(userOption.get) else Future.successful(None) //generator 3. found user and token. Update profile
deletedToken:Option[UserTokenKey] <- if(modifiedUser.isDefined) userTokenRepo.delete(UserTokenKey(UUID.fromString(token))) else Future.successful(None)
}
yield { //check if we have user and token and modified user here. If any is missing, return error else success
println("db query results tokenOption: "+tokenOption+", userOption: "+userOption+" : modifiedUserOption: "+modifiedUser+", deletedToken: "+deletedToken)
if(tokenOption.isDefined && userOption.isDefined && modifiedUser.isDefined && deletedToken.isDefined)
Redirect("http://localhost:9000/home"+";signup=success")//TODOM - pick from config
else
/*TODOM - when redirecting with error, can provide additional info why sign up failed*/
if(tokenOption.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else if(userOption.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else if(modifiedUser.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else //this shouldn't happen. Unexpected
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
}
result.recover { case x => {
println("Future failed in validateUserSession. Recovering. Returning Internal Server Error" + x)
//before this Redirect, the custom error handler sends json response
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
}
}
Custom error handler
class CustomHttpErrorHandler extends HttpErrorHandler {
def onClientError(request: RequestHeader, statusCode: Int, message: String) = {
println("client error: request "+request+", statusCode: "+statusCode+", message:"+message)
Future.successful(
Status(statusCode)(Json.toJson(JsonResultError(message)))
)
}
def onServerError(request: RequestHeader, exception: Throwable) = {
println("server error: request: "+request+", exception: "+exception.getMessage)
Future.successful(
InternalServerError(Json.toJson(JsonResultError(exception.getMessage)))
)
}
}
I am able to verify the isse as I see two debugs (one from custom error handler and other from recover)
server error: request: GET /ws/users/signup/312c9eaf-f27b-43c7-8dac-445a628c3be8, exception: bucket_id is not a column defined in this metadata
Future failed in validateUserSession. Recovering. Returning Internal Server Errorjava.lang.IllegalArgumentException: bucket_id is not a column defined in this metadata
I could try to check based on the exception received in custom error handler but I think it is too generic and might not be a good design approach.

It is possible that the error is happening before the Future is created so the code is throwing an exception that is handled by the default handler rather than being caught by the Future.
Specifically, the expression
userTokenRepo.findOne(UserTokenKey(UUID.fromString(token)))
is evaluated in the current thread, so any exception in this code will not be caught and will invoke the default error handler.
The solution is to compute this in a Try and process the error immediately if there is one.
This is what it might look like:
for {
tokenKey <- Future.fromTry(Try(UserTokenKey(UUID.fromString(token))))
tokenOption <- userTokenRepo.findOne(tokenKey)
userOption <- tokenOption.fold(Future.successful)(userRepo.findOne(_.userKeys)) //generator2. found token, look for corresponding user to which the token belongs
modifiedUser <- userOption.fold(Future.successful)(confirmSignupforUser) //generator 3. found user and token. Update profile
...
Any exception in this code will result in a failed Future which your recover code will handle.

Related

Abort (HTTP 500) request when exception in request's handleEnd()

I'm using vert.x-web to implement a small service. One of my handlers for the end of the request (set via context.request().endHandler()) throws this NullPointerException:
2018-09-02 20:54:35,125 - ERROR [vert.x-eventloop-thread-1] (ContextImpl.java:345) - lambda$wrapTask$2()
Unhandled exception
java.lang.NullPointerException
at (My handler class)
at io.vertx.core.http.impl.HttpServerRequestImpl.handleEnd(HttpServerRequestImpl.java:417)
at io.vertx.core.http.impl.Http1xServerConnection.handleEnd(Http1xServerConnection.java:482)
at io.vertx.core.http.impl.Http1xServerConnection.handleContent(Http1xServerConnection.java:477)
at io.vertx.core.http.impl.Http1xServerConnection.processMessage(Http1xServerConnection.java:458)
at io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:144)
at io.vertx.core.http.impl.HttpServerImpl$ServerHandlerWithWebSockets.handleMessage(HttpServerImpl.java:712)
at io.vertx.core.http.impl.HttpServerImpl$ServerHandlerWithWebSockets.handleMessage(HttpServerImpl.java:619)
at io.vertx.core.net.impl.VertxHandler.lambda$channelRead$1(VertxHandler.java:146)
at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:337)
at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:195)
at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:144)
Why doesn't this exception call my requests's exception handler? Why is it unhandled? I have the request's exception handler set to context.fail() (via context.request().exceptionHandler()). But it does not seem to have any effect.
Is there another exception handler I'm unaware of?
Edit: here is the minimal reproducing code:
Vertx vertx = Vertx.vertx();
Router router = Router.router(vertx);
router.route().handler(context -> {
context.request()
.exceptionHandler(context::fail)
.endHandler(nothing -> { throw new NullPointerException("null"); })
.handler(buffer -> {});
});
vertx.createHttpServer()
.requestHandler(router::accept)
.listen(8080);
Expected behavior: context.fail(e) gets called and the connection closes with HTTP ERROR 500.
Got behavior: context is not failed, connection "hangs".
The exceptionHandler applies to the HttpServerRequest object. The method is inherited from the ReadStream interface. This callback is invoked whenever a problem occurs in the Vert.x/Netty code handling HTTP requests, not user code.
If you want to execute some code before the actual request processing, I would suggest to register a route and invoke RoutingContext#next in the handler:
Router router = Router.router(vertx);
router.route("/somepath").handler(routingContext -> {
// Handler invoked first
// Execute pre-processing logic
// And then...
context.next();
});
router.route("/somepath").handler(routingContext -> {
// Handler invoked second
// Execute processing logic
});
Then any failure in pre-processing logic will be caught and managed normally by the router.
Two things need to be pointed out here:
The hang is caused by the response is not explicitly ended (see HttpServerResponse#end().
To handle exception happened during request handling, add failure handler at route level (see Route#failureHandler()). Handling exception on request will only caught exception when reading the stream.
For example:
Router router = Router.router(vertx);
router.route().failureHandler(handler -> handler.response().end());
router.route().handler(routingContext -> routingContext.request().endHandler(handler -> {
throw new NullPointerException("exception here!");
}));
vertx.createHttpServer().requestHandler(router::accept).listen(8085);

Play framework Scala run job in background

Is there any way I can trigger a job from the controller (to not to wait for its completion) and display the message to the user that job will be running in the background?
I have one controller method which takes quite long time to run. So I want to make that run offline and display the message to the user that it will be running in the background.
I tried Action.async as shown below. But the processing of the Future object is still taking more time and getting timed out.
def submit(id: Int) = Action.async(parse.multipartFormData) { implicit request =>
val result = Future {
//process the data
}
result map {
res =>
Redirect(routes.testController.list()).flashing(("success", s"Job(s) will be ruuning in background."))
}
}
You can also return a result without waiting for the result of the future in a "fire and forget" way
def submit(id: Int) = Action(parse.multipartFormData) { implicit request =>
Future {
//process the data
}
Redirect(routes.testController.list()).flashing(("success", s"Job(s) will be running in background."))
}
The docs state:
By giving a Future[Result] instead of a normal Result, we are able to quickly generate the result without blocking. Play will then serve the result as soon as the promise is redeemed.
The web client will be blocked while waiting for the response, but nothing will be blocked on the server, and server resources can be used to serve other clients.
You can configure your client code to use ajax request and display a Waiting for data message for some part of the page without blocking the rest of the web page from loading.
I also tried the "Futures.timeout" option. It seems to work fine. But I'm not sure its correct way to do it or not.
result.withTimeout(20.seconds)(futures).map { res =>
Redirect(routes.testController.list()).flashing(("success", s"Job(s) will be updated in background."))
}.recover {
case e: scala.concurrent.TimeoutException =>
Redirect(routes.testController.list()).flashing(("success", s"Job(s) will be updated in background."))
}

Chain Akka-http-client requests in a Stream

I would like to chain http request using akka-http-client as Stream. Each http request in a chain depends on a success/response of a previous requests and uses it to construct a new request. If a request is not successful, the Stream should return the response of the unsuccessful request.
How can I construct such a stream in akka-http?
which akka-http client level API should I use?
If you're making a web crawler, have a look at this post. This answer tackles a more simple case, such as downloading paginated resources, where the link to the next page is in a header of the current page response.
You can create a chained source - where one item leads to the next - using the Source.unfoldAsync method. This takes a function which takes an element S and returns Future[Option[(S, E)]] to determine if the stream should continue emitting elements of type E, passing the state to the next invocation.
In your case, this is kind of like:
taking an initial HttpRequest
producing a Future[HttpResponse]
if the response points to another URL, returning Some(request -> response), otherwise None
However, there's a wrinkle, which is that this will not emit a response from the stream if it doesn't contain a pointer to the next request.
To get around this, you can make the function passed to unfoldAsync return Future[Option[(Option[HttpRequest], HttpResponse)]]. This allows you to handle the following situations:
the current response is an error
the current response points to another request
the current response doesn't point to another request
What follows is some annotated code which outlines this approach, but first a preliminary:
When streaming HTTP requests to responses in Akka streams, you need to ensure that the response body is consumed otherwise bad things will happen (deadlocks and the like.) If you don't need the body you can ignore it, but here we use a function to convert the HttpEntity from a (potential) stream into a strict entity:
import scala.concurrent.duration._
def convertToStrict(r: HttpResponse): Future[HttpResponse] =
r.entity.toStrict(10.minutes).map(e => r.withEntity(e))
Next, a couple of functions to create an Option[HttpRequest] from an HttpResponse. This example uses a scheme like Github's pagination links, where the Links header contains, e.g: <https://api.github.com/...> rel="next":
def nextUri(r: HttpResponse): Seq[Uri] = for {
linkHeader <- r.header[Link].toSeq
value <- linkHeader.values
params <- value.params if params.key == "rel" && params.value() == "next"
} yield value.uri
def getNextRequest(r: HttpResponse): Option[HttpRequest] =
nextUri(r).headOption.map(next => HttpRequest(HttpMethods.GET, next))
Next, the real function we'll pass to unfoldAsync. It uses the Akka HTTP Http().singleRequest() API to take an HttpRequest and produce a Future[HttpResponse]:
def chainRequests(reqOption: Option[HttpRequest]): Future[Option[(Option[HttpRequest], HttpResponse)]] =
reqOption match {
case Some(req) => Http().singleRequest(req).flatMap { response =>
// handle the error case. Here we just return the errored response
// with no next item.
if (response.status.isFailure()) Future.successful(Some(None -> response))
// Otherwise, convert the response to a strict response by
// taking up the body and looking for a next request.
else convertToStrict(response).map { strictResponse =>
getNextRequest(strictResponse) match {
// If we have no next request, return Some containing an
// empty state, but the current value
case None => Some(None -> strictResponse)
// Otherwise, pass on the request...
case next => Some(next -> strictResponse)
}
}
}
// Finally, there's no next request, end the stream by
// returning none as the state.
case None => Future.successful(None)
}
Note that if we get an errored response, the stream will not continue since we return None in the next state.
You can invoke this to get a stream of HttpResponse objects like so:
val initialRequest = HttpRequest(HttpMethods.GET, "http://www.my-url.com")
Source.unfoldAsync[Option[HttpRequest], HttpResponse](
Some(initialRequest)(chainRequests)
As for returning the value of the last (or errored) response, you simply need to use Sink.last, since the stream will end either when it completes successfully or on the first errored response. For example:
def getStatus: Future[StatusCode] = Source.unfoldAsync[Option[HttpRequest], HttpResponse](
Some(initialRequest))(chainRequests)
.map(_.status)
.runWith(Sink.last)

spray and actor non deterministic tests

Helo,
at the beginning i wold like to apologize for my english :)
akka=2.3.6
spray=1.3.2
scalatest=2.2.1
I encountered strange behavior of teting routes, which asks actors in handleWith directive,
I've route with handleWith directive
pathPrefix("firstPath") {
pathEnd {
get(complete("Hello from this api")) ~
post(handleWith { (data: Data) =>{ println("receiving data")
(dataCalculator ? data).collect {
case Success(_) =>
Right(Created -> "")
case throwable: MyInternalValidatationException =>
Left(BadRequest -> s"""{"${throwable.subject}" : "${throwable.cause}"}""")
}
}})
}
}
and simple actor wchich always responds when receive object Data and has own receive block wrapped in LoggingReceive, so I should see logs when message is receiving by actor
and i test it using (I think simple code)
class SampleStarngeTest extends WordSpec with ThisAppTestBase with OneInstancePerTest
with routeTestingSugar {
val url = "/firstPath/"
implicit val routeTestTimeout = RouteTestTimeout(5 seconds)
def postTest(data: String) = Post(url).withJson(data) ~> routes
"posting" should {
"pass" when {
"data is valid and comes from the identified user" in {
postTest(correctData.copy(createdAt = System.currentTimeMillis()).asJson) ~> check {
print(entity)
status shouldBe Created
}
}
"report is valid and comes from the anonymous" in {
postTest(correctData.copy(createdAt = System.currentTimeMillis(), adid = "anonymous").asJson) ~> check {
status shouldBe Created
}
}
}
}
}
and behavior:
When I run either all tests in package (using Intellij Idea 14 Ultimate) or sbt test I encounter the same results
one execution -> all tests pass
and next one -> not all pass, this which not pass I can see:
1. fail becouse Request was neither completed nor rejected within X seconds ( X up tp 60)
2. system console output from route from line post(handleWith { (data: Data) =>{ println("receiving data"), so code in handleWith was executed
3. ask timeout exception from route code, but not always (among failed tests)
4. no logs from actor LoggingReceive, so actor hasn't chance to respond
5. when I rerun teststhe results are even different from the previous
Is there problem with threading? or test modules, thread blocking inside libraries? or sth else? I've no idea why it isn't work :(

ReactiveMongo: How to deal with database errors when inserting new documents

I've a MongoDB collection where I store User documents like this:
{
"_id" : ObjectId("52d14842ed0000ed0017cceb"),
"email": "joe#gmail.com",
"firstName": "Joe"
...
}
Users must be unique by email address, so I added an index for the email field:
collection.indexesManager.ensure(
Index(List("email" -> IndexType.Ascending), unique = true)
)
And here is how I insert a new document:
def insert(user: User): Future[User] = {
val json = user.asJson.transform(generateId andThen copyKey(publicIdPath, privateIdPath) andThen publicIdPath.json.prune).get
collection.insert(json).map { lastError =>
User(json.transform(copyKey(privateIdPath, publicIdPath) andThen privateIdPath.json.prune).get).get
}.recover {
throw new IllegalArgumentException(s"an user with email ${user.email} already exists")
}
}
In case of error, the code above throws an IllegalArgumentException and the caller is able to handle it accordingly. BUT if I modify the recover section like this...
def insert(user: User): Future[User] = {
val json = user.asJson.transform(generateId andThen copyKey(publicIdPath, privateIdPath) andThen publicIdPath.json.prune).get
collection.insert(json).map { lastError =>
User(json.transform(copyKey(privateIdPath, publicIdPath) andThen privateIdPath.json.prune).get).get
}.recover {
case e: Throwable => throw new IllegalArgumentException(s"an user with email ${user.email} already exists")
}
}
... I no longer get an IllegalArgumentException, but I get something like this:
play.api.Application$$anon$1: Execution exception[[IllegalArgumentException: DatabaseException['E11000 duplicate key error index: gokillo.users.$email_1 dup key: { : "giuseppe.greco#agamura.com" }' (code = 11000)]]]
... and the caller is no longer able to handle the exception as it should. Now the real questions are:
How do I handle the diverse error types (i.e. the ones provided by LastError) in the recover section?
How do I ensure the caller gets the expected exceptions (e.g. IllegalArgumentException)?
Finally I was able to manage things correctly. Here below is how to insert an user and handle possible exceptions with ReactiveMongo:
val idPath = __ \ 'id
val oidPath = __ \ '_id
/**
* Generates a BSON object id.
*/
protected val generateId = __.json.update(
(oidPath \ '$oid).json.put(JsString(BSONObjectID.generate.stringify))
)
/**
* Converts the current JSON into an internal representation to be used
* to interact with Mongo.
*/
protected val toInternal = (__.json.update((oidPath \ '$oid).json.copyFrom(idPath.json.pick))
andThen idPath.json.prune
)
/**
* Converts the current JSON into an external representation to be used
* to interact with the rest of the world.
*/
protected val toExternal = (__.json.update(idPath.json.copyFrom((oidPath \ '$oid).json.pick))
andThen oidPath.json.prune
)
...
def insert(user: User): Future[User] = {
val json = user.asJson.transform(idPath.json.prune andThen generateId).get
collection.insert(json).transform(
success => User(json.transform(toExternal).get).get,
failure => DaoServiceException(failure.getMessage)
)
}
The user parameter is a POJO-like instance with an internal representation in JSON – User instances always contain valid JSON since it is generated and validated in the constructor and I no longer need to check whether user.asJson.transform fails.
The first transform ensures there is no id already in the user and then generates a brand new Mongo ObjectID. Then, the new object is inserted in the database, and finally the result converted back to the external representation (i.e. _id => id). In case of failure, I just create a custom exception with the current error message. I hope that helps.
My experience is more with the pure java driver, so I can only comment on your strategy for working with mongo in general -
It seems to me that all you're accomplishing by doing the query beforehand is duplicating mongos uniqueness check. Even with that, you still have to percolate an exception upwards because of possible failure. Not only is this slower, but it's vulnerable to a race condition because the combination of your query + insert is not atomic. In that case you'd have
request 1: try to insert. email exists? false - Proceed with insert
request 2: try to insert. email exists? false - Proceed with insert
request 1: succeed
request 2: mongo will throw the database exception.
Wouldn't it be simpler to just let mongo throw the db exception and throw your own illegal argument if that happens?
Also, pretty sure the id will be generated for you if you omit it, and that there's a simpler query for doing your uniqueness check, if that's still the way you want to code it. At least in the java driver you can just do
collection.findOne(new BasicDBObject("email",someemailAddress))
Take a look at upsert mode of the update method (section "Insert a New Document if No Match Exists (Upsert)"): http://docs.mongodb.org/manual/reference/method/db.collection.update/#insert-a-new-document-if-no-match-exists-upsert
I asked a similar question a while back on reactivemongo's google group. You can have another case inside the recovery block to match a LastError object, query its error code, and handle the error appropriately. Here's the original question:
https://groups.google.com/forum/#!searchin/reactivemongo/alvaro$20naranjo/reactivemongo/FYUm9x8AMVo/LKyK01e9VEMJ