I need to query a RESTful service that always returns a JSON response. I need to contact it a few times, always with some more information that I learned from the previous request. I'm using Akka2, Scala, Jerkson and Spray-Can.
My current approach seems to work, but it looks ugly and requires nesting everything. I read that there should be some techniques available regarding chaining and such, but I couldn't figure out how to apply them to my current use-case.
Here is the code I'm talking about:
def discoverInitialConfig(hostname: String, bucket: String) = {
val poolResponse: Future[HttpResponse] =
HttpDialog(httpClient, hostname, 8091)
.send(HttpRequest(uri = "/pools"))
.end
poolResponse onComplete {
case Right(response) =>
log.debug("Received the following global pools config: {}", response)
val pool = parse[PoolsConfig](response.bodyAsString)
.pools
.find(_("name") == defaultPoolname)
.get
val selectedPoolResponse: Future[HttpResponse] =
HttpDialog(httpClient, hostname, 8091)
.send(HttpRequest(uri = pool("uri")))
.end
selectedPoolResponse onComplete {
case Right(response) =>
log.debug("Received the following pool config: {}", response)
println(response.bodyAsString)
case Left(failure) =>
log.error("Could not load the pool config! {}", failure)
}
case Left(failure) =>
log.error("Could not load the global pools config! {}", failure)
}
I think you can see the pattern. Contact the REST service, read it, on success parse it into a JSON case class, extract information out and then do the next call.
My structure here is only two-levels deep but I need to add a third level as well.
Is there a technique available to improve this for better readability or can I only stick with this? If you need any further information I'm happy to provide it. You can see the full code here: https://github.com/daschl/cachakka/blob/f969d1f56a4c90a929de9c7ed4e4a0cccea5ba70/src/main/scala/com/cachakka/cluster/actors/InitialConfigLoader.scala
Thanks,
Michael
HttpDialog seems to cover your use case exactly.
I think I found a reasonable shortcut by using the provided reply method from spray-can.
def discoverInitialConfig(hostname: String, bucket: String) = {
val poolResponse: Future[HttpResponse] =
HttpDialog(httpClient, hostname, 8091)
.send(HttpRequest(uri = "/pools"))
.reply(response => {
log.debug("Received the following global pools config: {}", response)
val selectedPool = parse[PoolsConfig](response.bodyAsString)
.pools.find(_("name") == defaultPoolname).get
HttpRequest(uri = selectedPool("uri"))
})
.reply(response => {
log.debug("Received the following pool config: {}", response)
println(response.bodyAsString)
HttpRequest(uri = "/")
})
.end
}
If this is the best available approach, I'll mark it as "answered" but I'm eager to get actual replies from people who know this stuff much better than me.
Related
Not sure if I'm getting this whole routing DSL thing right but here's the question. I want to do a post to external service such as:
val post = pathPrefix("somePath") {
post {
//get the response mapped to my Output object
}
}
Then I want the response (which is a Json) to be mapped to an object matching the fields for example Output (assuming I have my JsonProtocol set up). How is this done?
You are using HTTP server directives to "retrieve" something "externally". This is what typically an HTTP client does.
For this sort of things, you can use akka http client api.
For example:
val response = Http().singleRequest(HttpRequest(uri = "http://akka.io"))
response onComplete {
case Success(res) =>
val entity = Unmarshal(res.entity).to[YourDomainObject]
// use entity here
case Failure(ex) => // do something here
}
However, this requires some Unmarshaller (to deserialize the received json). Take also a look at Json Support, as it helps you define marshallers easily:
case class YourDomainObject(id: String, name: String)
implicit val YourDomainObjectFormat = jsonFormat2(YourDomainObject)
I think what you are trying to ask is how to get the body i.e in JSOn format to the Case class that you have
Here is a quick example:
path("createBot" / Segment) { tag: String =>
post {
decodeRequest {
entity(as[CaseClassName]) { caseclassInstance: CaseClassName =>
val updatedAnswer = doSomeStuff(caseclassInstance)
complete {
"Done"
}
}
}
You can find more detailed example from here : https://github.com/InternityFoundation/Stackoverflowbots/blob/master/src/main/scala/in/internity/http/RestService.scala#L56
I hope it answers your question.
Right now I am using akka-stream and akka-HTTP to build a file streaming API. As such I am injecting a streaming source into an entity to have data streamed directly to the HTTP client like so:
complete(HttpEntity(ContentTypes.`application/octet-stream`, source))
However, if for some reason the stream fails, the connection gets closed by akka-http without further explanation or logging.
I would need 2 things:
How can I get the exception logs?
How can I notify my client with a message before closing the connection?
Thank you
As mentioned in comment HTTP protocol does not allow to signal error to the client side.
As to logging:
For me it boils down to missing proper access log directive in akka http.
In my current project we have decorator which register onComplete handler for http entity before giving it to akka http for rendering.
private def onResponseStreamEnd(response: HttpResponse)(action: StatusCode => Unit): HttpResponse =
if (!response.status.allowsEntity() || response.entity.isKnownEmpty()) {
action(response.status)
response
} else {
val dataBytes =
onStreamEnd(response.entity) { result =>
val overallStatusCode =
result match {
case Success(_) =>
response.status
case Failure(e) =>
logger.error(e, s"error streaming response [${e.getMessage}]")
StatusCodes.InternalServerError
}
action(overallStatusCode)
}
response.withEntity(response.entity.contentLengthOption match {
case Some(length) => HttpEntity(response.entity.contentType, length, dataBytes)
case None => HttpEntity(response.entity.contentType, dataBytes)
})
}
private def onStreamEnd(entity: HttpEntity)(onComplete: Try[Done] ⇒ Unit): Source[ByteString, _] =
entity.dataBytes.alsoTo { Sink.onComplete(onComplete) }
Usage:
complete(onResponseStreamEnd(HttpResponse(StatusCodes.OK, HttpEntity(ContentTypes.`application/octet-stream`, source))){ statusCode => .... })
Similar approach but using custom graph stage you can find here
I have a controller that exposes a method as a route. In this method, I call a long running computation that returns a Future[SomeType].
I now have the following:
def compute(id: String) = Action.async { request =>
val result: Future[SomeType] = compute(id)
result.map(value => Ok(transform(value, id)))
}
So far this is just the happy path. What if compute(id) results in a Failure? How to handle that? I could wrap the whole thing in a Try block, but is there a better alternative? Any suggestions?
We usually use the following pattern:
def compute(id: String) = Action.async { request =>
val result: Future[SomeType] = compute(id)
result.map(value => Ok(transform(value, id)))
.recover { case ex =>
Logger.error("Something went wrong", ex)
InternalServerError
}
}
This way the HTTP response code will be 500 INTERNAL SERVER ERROR, so the caller will be informed. You may also want to add validation on the parameters of the request and return a 400 BAD REQUEST etc.
I am using spray to serve an api. I am trying to create a directive to add a list of headers to all responses, including rejections and failures. I have tried the following but both only work for successful responses:
val impl1: Directive0 = respondWithSingletonHeaders(myHeaderList)
val impl2: Directive0 = mapRequestContext { ctx =>
ctx.withHttpResponseHeadersMapped { headers =>
myHeaderList ::: headers
}
}
Is there an equivalent construct that could work for ALL responses? I suppose I can define custom error handlers, but, correct me if I am wrong, they are expected to work by calling ctx.complete(...), since it a side effect, I think I will have to override every failure case. I haven't found a place where I can simply map HttpResponses to add the headers.
An exception will bubble up the route as an exception, until it reaches an exception handler. And you can't add headers to an exception. But you can re-use your directives in your exception handlers. You can avoid repetition by making an implicit ExceptionHandler at the runRoute level, use your directive to add headers for the general case (all exceptions) and delegate the actual exception-to-response mapping to an inner pattern match:
implicit def exceptionHandler: ExceptionHandler = ExceptionHandler {
case exception => impl1 {
exception match {
case e: IllegalArgumentException => ctx =>
ctx.complete(BadRequest)
...
case e: Exception => ctx =>
ctx.complete(InternalServerError)
}
}
}
I've been using the Databinder Dispatch library in a client for a simple REST-ish API. I know how to detect if I get an HTTP response with an error status:
Http x (request) {
case (200, _, _, content) => successResult(content())
case (404, _, _, _) => notFoundErrorResult
case (_, _, _, _) => genericErrorResult
}
But how can I distinguish an error response from a failure to get any response at all, because of an invalid domain or failure to connect? And is there any way to implement a timeout while still using synchronous semantics? If there's anything relevant in the API, I've missed it.
There is also a more elegant way to configure client using Http.configure method which receives Builder => Builder function as an argument:
val http = Http.configure(_.setAllowPoolingConnection(true).setConnectionTimeoutInMs(5000))
The Periodic Table tells us that >! sets up an exception listener and a recent mailing list thread explains how to set a timeout.
All together, then, you might do something like:
val http = new dispatch.Http {
import org.apache.http.params.CoreConnectionPNames
client.getParams.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 2000)
client.getParams.setParameter(CoreConnectionPNames.SO_TIMEOUT, 5000)
}
http(req >! {
case e => // ...
})
Note that I haven't tested this...
In case you are using Dispatch reboot (with AsyncHttpClient as the underlying library) this is how you'd set the client configuration:
val myHttp = new dispatch.Http {
import com.ning.http.client._
val builder = new AsyncHttpClientConfig.Builder()
builder.setCompressionEnabled(true)
.setAllowPoolingConnection(true)
.setRequestTimeoutInMs(5000)
override lazy val client = new AsyncHttpClient(builder.build())
}
and then just use this new object as you'd otherwise use http:
myHttp((url(baseUrl) <<? args) OK as.xml.Elem).either