How do I catch downstream exceptions such as java.io.IOException when returning responses?
I tried using CancellationStrategy.FailStage attribute but stage still completes successfully when using watchTermination?
// my.app.WebServer
Http()
.newServerAt(config.server.host, config.server.port)
.connectionSource()
.log("connection", _.remoteAddress.toString)
.addAttributes(Attributes.logLevels(onElement = Attributes.LogLevels.Debug,
onFinish = Attributes.LogLevels.Debug,
onFailure = Attributes.LogLevels.Error))
.addAttributes(Attributes(CancellationStrategy(CancellationStrategy.FailStage)))
.recover {
case ex =>
log.error("Connection error", ex)
throw ex
}
.runForeach(_.handleWithAsyncHandler(route))
// my.app.Api
private def respondWith(httpResponse: HttpResponse): Route = {
complete {
httpResponse
.withEntity(HttpEntity(
httpResponse.entity.contentType,
httpResponse.entity.dataBytes
.log("response")
.addAttributes(Attributes.logLevels(onElement = Attributes.LogLevels.Off,
onFinish = Attributes.LogLevels.Info,
onFailure = Attributes.LogLevels.Error))
.addAttributes(Attributes(CancellationStrategy(CancellationStrategy.FailStage)))
.watchTermination()((_, future) =>
future.onComplete {
case Success(_) => log.info("Response completed")
case Failure(ex) => log.error("Response failed", ex)
})
))
}
}
// Log
2021-12-13T08:11:55,346 DEBUG [63] akka.io.TcpListener: New connection accepted
2021-12-13T08:11:55,348 DEBUG [56] akka.stream.Materializer: [connection] Element: /0:0:0:0:0:0:0:1:51294
2021-12-13T08:11:55,353 DEBUG [56] ...
2021-12-13T08:11:57,675 DEBUG [52] akka.io.TcpIncomingConnection: Closing connection due to IO error java.io.IOException: Broken pipe
2021-12-13T08:11:57,911 INFO [17] my.app.Api$: Response completed
2021-12-13T08:11:57,912 INFO [17] akka.stream.Materializer: [response] Downstream finished, cause: SubscriptionWithCancelException$NoMoreElementsNeeded$: null
Related
I am using CloseableHttpAsyncClient for downloading image with socketreadtimemout of 10 sec , connect timeout 5sec and doing a retry if it fails with sockettimeout and Timeout exception.
Now when the url is not accessible(404) then instead of resource not found exception it is failing with socket timeout and doing a retry again.So in my case for this invalid url also we are trying again and it adds up to the latency(~10+10=20 sec).
Below is the code snippet
Future<HttpResponse> httpResponseFuture = asyncCloseableHttpClient.execute(httpUriRequest, null);
try {
return httpResponseFuture.get(10000, TimeUnit.MILLISECONDS);
} catch (ExecutionException e) {
Throwable cause = e.getCause() != null ? e.getCause() : e;
if (cause instanceof ConnectException) {
throw new DownloadConnectionException("ConnectionException " + cause, DOWNLOAD_FAILED, cause);
}
if (cause instanceof SocketTimeoutException) {
throw new DownloadTimeoutException(DOWNLOAD_TIMEOUT_EXCEPTION);
}
if (cause instanceof ConnectionClosedException) {
throw new DownloadConnectionClosedException("ConnectionClosedException " + DOWNLOAD_FAILED, cause);
}
if(cause instanceof UnsupportedCharsetException) {
throw new BadRequestException("Image download failed with UnsupportedCharsetException " + cause,
INVALID_CONTENT_TYPE_EXCEPTION, cause);
}
} catch (TimeoutException e) {
throw new DownloadTimeoutException(DOWNLOAD_TIMEOUT_EXCEPTION);
}
This the config values for CloseableHttpAsyncClient
RequestConfig config = RequestConfig.custom()
.setSocketTimeout(10000)
.setConnectTimeout(5000)
.setRedirectsEnabled(true)
.setMaxRedirects(3)
.setStaleConnectionCheckEnabled(false) // never set this to true, 30 ms extra per request
.setProxyPreferredAuthSchemes(Arrays.asList(AuthSchemes.BASIC))
.build();
This is retry config
RetryConfig.custom().maxAttempts(downloadAttempts).retryOnException(e -> ( e instanceof DownloadTimeoutException) ) .waitDuration(Duration.ofMillis(30)).build();
To give more context why I am setting the socketreadtimemout of 10 sec because let's say for some bigger image download it may fail 1st time,so in that case retry is valid scenario but not in the resource not found case.
Can we do anything to make resource not found/invalid url fail fast so that it wont fail with sockettimeout exception.
I am using Akka Http (v. 10.1.10) to create a client with proxy.
Each time the response is any other than successful, I get an error instead of a proper response entity:
akka.http.impl.engine.client.ProxyConnectionFailedException: The HTTP(S) proxy rejected to open a connection to hahahahahhahaahahhahaadsfsd.com:80 with status code: 503 Service Unavailable
at akka.http.impl.engine.client.HttpsProxyGraphStage$$anon$1$$anon$4.onPush(HttpsProxyGraphStage.scala:143)
at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:523)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:409)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:606)
at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:485)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:581)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:749)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:764)
at akka.actor.Actor.aroundReceive(Actor.scala:539)
at akka.actor.Actor.aroundReceive$(Actor.scala:537)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:671)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:612)
at akka.actor.ActorCell.invoke(ActorCell.scala:581)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268)
at akka.dispatch.Mailbox.run(Mailbox.scala:229)
at akka.dispatch.Mailbox.exec(Mailbox.scala:241)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
In the Akka Http core code, I found that it is the implementation of the handler for proxy that forces the error on purpose if the response is any other than successful:
case Connecting =>
val proxyResponse = grab(bytesIn)
parser.parseBytes(proxyResponse) match {
case NeedMoreData =>
pull(bytesIn)
case ResponseStart(_: StatusCodes.Success, _, _, _, _) =>
var pushed = false
val parseResult = parser.onPull()
require(parseResult == ParserOutput.MessageEnd, s"parseResult should be MessageEnd but was $parseResult")
parser.onPull() match {
// NeedMoreData is what we emit in overridden `parseMessage` in case input.size == offset
case NeedMoreData =>
case RemainingBytes(bytes) =>
push(sslOut, bytes) // parser already read more than expected, forward that data directly
pushed = true
case other =>
throw new IllegalStateException(s"unexpected element of type ${other.getClass}")
}
parser.onUpstreamFinish()
log.debug(s"HTTP(S) proxy connection to {}:{} established. Now forwarding data.", targetHostName, targetPort)
state = Connected
if (isAvailable(bytesOut)) pull(sslIn)
if (isAvailable(sslOut)) pull(bytesIn)
case ResponseStart(statusCode, _, _, _, _) =>
failStage(new ProxyConnectionFailedException(s"The HTTP(S) proxy rejected to open a connection to $targetHostName:$targetPort with status code: $statusCode"))
case other =>
throw new IllegalStateException(s"unexpected element of type $other")
}
I am wondering what is the reason for such implementation, if someone knows? And how to work it around to get a response entity instead of the error when response from the server is not successful?
Try recover function to capture and return response entity you want.
Http().singleRequest(HttpRequest(
uri = url
)).map(response => {
response
}) recover {
case e: Exception => {
e.printStackTrace()
// anything you want to return
HttpResponse(StatusCodes.OK)
}
}
I call an API to get a zip file response. The API responds correctly but I am unable to get the byte array from response because the future that should complete on getting the ByteString never completes:
val authorization = akka.http.javadsl.model.headers.Authorization.basic("xxxxx", "xxxxxx")
val query = Map("fed" -> "xxxx", "trd" -> "yyy", "id" -> "zzz")
val request = HttpRequest(HttpMethods.GET, Uri("https://xxxx.yyyy.com/ggg/ttt.php").withQuery(Query(params = query))).addHeader(authorization)
val responseFut = http.singleRequest(request)
responseFut1.map(response => {
println("*******************************")
println(response)
response.status match {
case akka.http.javadsl.model.StatusCodes.OK => {
println("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^" + response._3)
val entityFut = response.entity.toStrict(60.seconds)
val byteStringFut = entityFut.flatMap(entity => {
entity.dataBytes.runFold(ByteString.empty)(_ ++ _)
})
println("#############################")
try {
byteStringFut.map(x => {
//this never prints =======================================problem
println("----------------------------" + x.toArray[Byte])
})
}catch{
case e: Exception => println("Error: " + e)
}
}
case _ => {}
}
})
If I print out the response this is what it looks like:
*******************************
HttpResponse(200 OK,List(Date: Fri, 08 Sep 2017 20:58:43 GMT, Server: Apache/2.4.18 (Ubuntu), Content-Disposition: attachment; filename="xxxxx.zip", Pragma: public, Cache-Contr
ol: public, must-revalidate, Content-Transfer-Encoding: binary),HttpEntity.Chunked(application/x-zip),HttpProtocol(HTTP/1.1))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^HttpEntity.Chunked(application/x-zip)
#############################
So response is coming back fine, but I still cannot get the binary data for the zip file.
We use akka-http in other places to call APIs that return json response and this approach seems to work fine there.
Why doesnt it work here? What am I doing wrong?
Any advice is appreciated.
Thanks.
Update
Adding byteStringFut.failed.foreach(println(_)) shows this exception: akka.http.scaladsl.model.EntityStreamException: HTTP chunk size exceeds the configured limit of 1048576 bytes
It looks like something went wrong and an exception was thrown in the async computation. You can check the exception by inspecting Future in the following way:
byteStringFut.failed.foreach(println)
In my repository function, I'm reading a User, then updating that user:
def update(u: User): Future[Int] = {
this.read(u.id).flatMap {
case Some(existingUser) =>
db.run(
userTable
.filter(_.id === user.id)
.update(user.copy(createdDate = existingUser.createdDate)))
//case None => throw new NotFoundException(); // does this exception exist in spray/akka?
}
}
I'd like to throw some sort of exception here when the user is not found, so that spray/akka will know that exception means to return HTTP 404 Not Found.
Does spray/akka contain some sort of NotFoundException that I can manually throw?
You can throw any exception and then configure an exception handler to convert the exception to 404 response.
Does it have to be an exception, or could you use this:
case None => HttpResponse(StatusCodes.NotFound)
I'm trying to limit life of observable by timeout:
def doLongOperation() = {
Thread.sleep(duration)
"OK"
}
def firstStep = Observable.create(
(observer: Observer[String]) => {
observer.onNext(doLongOperation())
observer.onCompleted()
Subscription()
}
)
firstStep
.timeout(1 second)
.subscribe(
item => println(item),
throwable => throw throwable,
() => println("complete")
)
I would like to distinguish between following results:
Observable finished by timeout, no result obtained
Exception thrown during execution
Execution finished successfully, return value
I can process cases 2 and 3 with no problem in partials onNext and onError, but how do I detect if observable finished by timeout?
One more thing: I've never get into block onComplete, though there is a call to obeserver.onCompleted() in my code. Why?
If a timeout happens, that TimeoutException is emitted on the computation thread where a throw throwable is ends up being ignored and your main thread won't and can't see it. You can add toBlocking after the timeout so any exception will end up on the same thread:
firstStep
.timeout(1 second)
.toBlocking()
.subscribe(
item => println(item),
throwable => println(throwable),
() => println("complete")
)
TimeoutException gets thrown indeed. The problem was caused by using wrong libraries. I had "com.netflix.rxjava" in my dependencies, instead of "io.reactivex"