Here is my simple routing application:
object Main extends App with SimpleRoutingApp {
implicit val system = ActorSystem("my-system")
startServer(interface = "0.0.0.0", port = System.getenv("PORT").toInt) {
import format.UsageJsonFormat._
import spray.httpx.SprayJsonSupport._
path("") {
get {
complete("OK")
}
} ~
path("meter" / JavaUUID) {
meterUUID => pathEnd {
post {
entity(as[Usage]) {
usage =>
// execute some logic asynchronously
// do not wait for the result
complete("OK")
}
}
}
}
}
}
What I want to achieve is to execute some logic asynchronously in my path directive, do not wait for the result and return immediately HTTP 200 OK.
I am quite new to Scala and spray and wondering if there is any spray way to solve this specific problem. Otherwise I would go into direction of creating Actor for every request and letting it to do the job. Please advice.
There's no special way of handling this in spray: simply fire your async action (a method returning a Future, a message sent to an actor, whatever) and call complete right after.
def doStuffAsync = Future {
// literally anything
}
path("meter" / JavaUUID) { meterUUID =>
pathEnd {
post {
entity(as[Usage]) { usage =>
doStuffAsync()
complete("OK")
}
}
}
}
Conversely, if you need to wait for an async action to complete before sending the response, you can use spray-specific directives for working with Futures or Actors.
Related
I've been trying to perform a test that uses a mock Http server to respond and a function that returns a Future[String] or an Exception if the Http server response isn't 200.
I'm trying to achieve a test without using Awaits, but instead AsyncFunSuiteLike.
However the following test seems impossible to resolve without doing it synchronously:
test("Error responses") {
Future.sequence {
NanoHTTPD.Response.Status.values().toList.filter(status => status.getRequestStatus >= 400).map {
status => {
httpService.setStatusCode(status)
val responseBody = s"Request failed with status $status"
httpService.setResponseContent(responseBody)
val errorMessage = s"Error response (${status.getRequestStatus}) from http service: $responseBody"
recoverToExceptionIf[ServiceException] {
myObject.httpCall("123456")
}.map {
ex => assert(ex.getMessage === errorMessage)
}
}
}
}.map(assertions => assert(assertions.forall(_ == Succeeded)))
}
Basically the problem is that when the Futures are tested, the NanoHTTPD is set to the last valued set in the map, so all ex.getMessage are the same. If I run those status codes one by one I do get the desired results, but, is there a way to perform all this in one single Async test?
From the looks of it, NanoHTTPD is stateful, so you have a race between the .set... calls and the .httpCall.
If you can spin up a new httpService within each Future, then you should be able to parallelize the tests (unless the state in question would be shared across instances, in which case you're likely out of luck).
So you'd have something like (replace Status with the type of status in your code and HTTPService with the type of httpService):
// following code composed on the fly and not run through the compiler...
def spinUpHTTPService(status: Status, body: String): Future[HTTPService] = {
// insert the code outside of the test which creates httpService
httpService.setStatusCode(status)
httpService.setResponseContent(body)
httpService
}
test("Error responses") {
Future.sequence(
NanoHTTPD.Response.Status.values().toList.filter(status => status.getRequestStatus >= 400).map { status =>
spinUpHTTPService(status, s"Request failed with status $status")
.flatMap { httpService =>
val errorMessage = s"Error response (${status.getRequestStatus}) from http service: $responseBody"
recoverToExceptionIf[ServiceException] {
myObject.httpCall("123456")
} map {
ex => assert(ex.getMessage === errorMessage)
}
} // Future.flatMap
} // List.map
).map { assertions => assertions.forAll(_ == Succeeded) }
}
I am building an application using Tumblr's new Colossus framework (http://tumblr.github.io/colossus/). There is still limited documentation on it (and the fact that I'm still very new to Akka doesn't help), so I was wondering if someone could chime in on whether my approach is correct.
The application is simple and consists of two key components:
A thin web service layer that will queue tasks into Redis
A background worker which will poll the same Redis instance for available tasks and process them as they become available
I made a simple example to demonstrate that my concurrency model will work (and it does), which I posted below. However, I would like to make sure that there is not a more idiomatic way to do this.
import colossus.IOSystem
import colossus.protocols.http.Http
import colossus.protocols.http.HttpMethod.Get
import colossus.protocols.http.UrlParsing._
import colossus.service.{Callback, Service}
import colossus.task.Task
object QueueProcessor {
implicit val io = IOSystem() // Create separate IOSystem for worker
Task { ctx =>
while(true) {
// Below code is for testing purposes only. This is where the Redis loop will live, and will use a blocking call to get the next available task
Thread.sleep(5000)
println("task iteration")
}
}
def ping = println("starting") // Method to launch this processor
}
object Main extends App {
implicit val io = IOSystem() // Primary IOSystem for the web service
QueueProcessor.ping // Launch worker
Service.serve[Http]("app", 8080) { ctx =>
ctx.handle { conn =>
conn.become {
case req#Get on Root => Callback.successful(req.ok("Here"))
// The methods to add tasks to the queue will live here
}
}
}
}
I tested the above model and it works. The background loop continues running while the service happily accepts requests. But, I think that there might be a better way to do this with workers (nothing found in documentation), or perhaps Akka Streams?
I got it working with something that seems semi-idiomatic to me. However, new answers & feedback are still welcomed!
class Processor extends Actor {
import scala.concurrent.ExecutionContext.Implicits.global
override def receive = {
case "start" => self ! "next"
case "next" => {
Future {
blocking {
// Blocking call here to wait on Redis (BRPOP/BLPOP)
self ! "next"
}
}
}
}
}
object Main extends App {
implicit val io = IOSystem()
val processor = io.actorSystem.actorOf(Props[Processor])
processor ! "start"
Service.serve[Http]("app", 8080) { ctx =>
ctx.handle { conn =>
conn.become {
// Queue here
case req#Get on Root => Callback.successful(req.ok("Here\n"))
}
}
}
}
Helo,
at the beginning i wold like to apologize for my english :)
akka=2.3.6
spray=1.3.2
scalatest=2.2.1
I encountered strange behavior of teting routes, which asks actors in handleWith directive,
I've route with handleWith directive
pathPrefix("firstPath") {
pathEnd {
get(complete("Hello from this api")) ~
post(handleWith { (data: Data) =>{ println("receiving data")
(dataCalculator ? data).collect {
case Success(_) =>
Right(Created -> "")
case throwable: MyInternalValidatationException =>
Left(BadRequest -> s"""{"${throwable.subject}" : "${throwable.cause}"}""")
}
}})
}
}
and simple actor wchich always responds when receive object Data and has own receive block wrapped in LoggingReceive, so I should see logs when message is receiving by actor
and i test it using (I think simple code)
class SampleStarngeTest extends WordSpec with ThisAppTestBase with OneInstancePerTest
with routeTestingSugar {
val url = "/firstPath/"
implicit val routeTestTimeout = RouteTestTimeout(5 seconds)
def postTest(data: String) = Post(url).withJson(data) ~> routes
"posting" should {
"pass" when {
"data is valid and comes from the identified user" in {
postTest(correctData.copy(createdAt = System.currentTimeMillis()).asJson) ~> check {
print(entity)
status shouldBe Created
}
}
"report is valid and comes from the anonymous" in {
postTest(correctData.copy(createdAt = System.currentTimeMillis(), adid = "anonymous").asJson) ~> check {
status shouldBe Created
}
}
}
}
}
and behavior:
When I run either all tests in package (using Intellij Idea 14 Ultimate) or sbt test I encounter the same results
one execution -> all tests pass
and next one -> not all pass, this which not pass I can see:
1. fail becouse Request was neither completed nor rejected within X seconds ( X up tp 60)
2. system console output from route from line post(handleWith { (data: Data) =>{ println("receiving data"), so code in handleWith was executed
3. ask timeout exception from route code, but not always (among failed tests)
4. no logs from actor LoggingReceive, so actor hasn't chance to respond
5. when I rerun teststhe results are even different from the previous
Is there problem with threading? or test modules, thread blocking inside libraries? or sth else? I've no idea why it isn't work :(
I'd like to be able to send back a response to the client before I do my logging/cleanup for a request.
In play 1.x this was possible with the #Finally annotation. I've read through some posts that say that those annotations were replaced by action composition, but I'm unclear how to emulate the #Finally annotation using it.
It seems to me that the response will only be returned after all the logic in my custom actions has completed.
Have I missed something, or is there no way to do this in Play 2.0?
[EDIT FOR CLARITY]
In other words, I want to be able to run logic after I receive a request and send a response. So I'd like to be able to construct a timeline of the form:
Client sends a request to my server
Server sends back a 200 response, which the client receives
The server does additional processing, logging, etc
In play 1.x I believe I could annote my additional processing logic with a #Finally and have it work like I want it to.
Action composition is not sufficient to do the job, but Action composition + Future, or Action composition + Actors are good ways to achieve this.
Action Composition + Future
Generate your Response, launch your logging/processing in an async context and, in parallel, send the result.
def LoggedAction(f: Request[AnyContent] => Result) = Action { request =>
val result = f(request)
concurrent.future(myLogAction(request, result))
result
}
Action composition + Actors
It's a cleaner way to achieve that. As in the previous case, generate your response, send logging/processing event to your(s) actor(s), and in parallel, send the result.
import play.api._
import play.api.mvc._
import play.libs._
import akka.actor._
object Application extends Controller with Finally {
def index = LoggedAction { r =>
Ok(views.html.index("Your new application is ready."))
}
}
trait Finally {
self: Controller =>
lazy val logActor = Akka.system.actorOf(Props(new LogActor), name = "logActor")
def LoggedAction(f: Request[AnyContent] => Result) = Action { request =>
val result = f(request) // Generate response
logActor ! LogRequest(request) // Send log event to LogActor
println("-> now send page to client")
result
}
case class LogRequest(request: Request[AnyContent])
class LogActor extends Actor {
def receive = {
case LogRequest(req) => {
println(req.host)
// ....
}
}
}
}
// Console
-> now send page to client
127.0.0.1:9000
So I want to write some network code that appears to be blocking, without actually blocking a thread. I'm going to send some data out on the wire, and have a 'queue' of responses that will come back over the network. I wrote up a very simple proof of concept, inspired by the producer/consumer example on the actor tutorial found here: http://www.scala-lang.org/node/242
The thing is, using receive appears to take up a thread, and so I'm wondering if theres anyway to not take up a thread and still get the 'blocking feel'. Heres my code sample:
import scala.actors.Actor._;
import scala.actors.Actor;
case class Request(val s:String);
case class Message(val s:String);
class Connection {
private val act:Actor = actor {
loop {
react {
case m:Message => receive { case r:Request => reply { m } }
}
}
}
def getNextResponse(): Message = {
return (act !? new Request("get")).asInstanceOf[Message];
}
//this would call the network layer and send something over the wire
def doSomething() {
generateResponse();
}
//this is simulating the network layer getting some data back
//and sending it to the appropriate Connection object
private def generateResponse() {
act ! new Message("someData");
act ! new Message("moreData");
act ! new Message("even more data");
}
}
object runner extends Application {
val conn = new Connection();
conn.doSomething();
println( conn.getNextResponse());
println(conn.getNextResponse());
println(conn.getNextResponse());
}
Is there a way to do this without using the receive, and thereby making it threadless?
You could directly rely on react which should block and release the thread:
class Connection {
private val act:Actor = actor {
loop {
react {
case r:Request => reply { r }
}
}
}
[...]
I expect that you can use react rather than receive and not have actors take up threads like receive does. There is thread on this issue at receive vs react.