Hi i'm creating a TCP server in Scala with the lib akka. I want to implement a restart of the tcp socket if my client doesn't send any data in a set interval. I tried with idleTimeout and a SupervisionStrategy but i can't catch the TimeoutException. On the client i see the log "Closing connection due to IO error java.io.IOException: Connection reset by peer"..
How i can resolve that? And restart the stream??
object TCPServer {
def serverLogic(connection: IncomingConnection) (implicit system: ActorSystem): Flow[ByteString, ByteString, NotUsed] = {
val converter: Flow[ByteString, String, NotUsed] = Flow[ByteString].map { (bytes: ByteString) =>
val message = bytes.utf8String
Logging.getLogger(system,this.getClass).debug(s"server received message $message")
message
}
val httpOut: Flow[String, String, NotUsed] = Flow[String].map { string =>
val answer: String = s"hello"
answer
}
val responder: Flow[String, ByteString, NotUsed] = Flow[String].map { string =>
val answer: String = s"Server responded with message [$string]"
ByteString(answer)
}
Flow[ByteString]
.idleTimeout(Duration.apply(1,"minutes"))
.via(converter)
.via(httpOut)
.via(responder)
}
def server(address: String, port: Int)(implicit system: ActorSystem) : Unit = {
val decider: Supervision.Decider = { e =>
LoggerFactory.getLogger(this.getClass).error("Failed ", e)
Supervision.Restart
}
val log = Logging.getLogger(system, this)
import system.dispatcher
val materializerSettings = ActorMaterializerSettings(system).withSupervisionStrategy(decider)
implicit val materializer = ActorMaterializer(materializerSettings)(system)
val connectionHandler: Sink[IncomingConnection, Future[Done]] = Sink.foreach[Tcp.IncomingConnection] { (conn: IncomingConnection) =>
log.debug(s"incomig connection from ${conn.remoteAddress}")
conn.handleWith(serverLogic(conn))
}
val incomingConnections: Source[IncomingConnection, Future[ServerBinding]] = Tcp().bind(address, port)
val binding: Future[ServerBinding] = incomingConnections.to(connectionHandler).run()
binding.onComplete {
case Success(b) =>
log.info("Server started, listening on: " + b.localAddress)
case Failure(e) =>
log.error(s"Server could not bind to $address:$port: ${e.getMessage}")
system.terminate()
}
}
}
Take a look to the doc. available in the akka stream webbsite: https://doc.akka.io/docs/akka/2.6.0/stream/stream-error.html. With this RestartSource you can apply a policy in order to restart TCP server.
How can I send elements/messages to an Akka Sink from an Akka HTTP route? My HTTP route still needs to return a normal HTTP response.
I imagine this requires a stream branch/junction. The normal HTTP routes are flows from HttpRequest -> HttpResponse. I would like to add a branch/junction so that HttpRequests can trigger events to my separate sink as well as generate the normal HttpResponse.
Below is a very simple single route akka-http app. For simplicity, I'm using a simple println sink. My production use case, will obviously involve a less trivial sink.
def main(args: Array[String]): Unit = {
implicit val actorSystem = ActorSystem("my-akka-http-test")
val executor = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()(actorSystem)
// I would like to send elements to this sink in response to HTTP GET operations.
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val route: akka.http.scaladsl.server.Route =
path("hello" / Segment) { p =>
get {
// I'd like to send a message to an Akka Sink as well as return an HTTP response.
complete {
s"<h1>Say hello to akka-http. p=$p</h1>"
}
}
}
val httpExt: akka.http.scaladsl.HttpExt = Http(actorSystem)
val bindingFuture = httpExt.bindAndHandle(RouteResult.route2HandlerFlow(route), "localhost", 8080)
println("Server online at http://localhost:8080/")
println("Press RETURN to stop...")
scala.io.StdIn.readLine()
bindingFuture
.flatMap(_.unbind())(executor) // trigger unbinding from the port
.onComplete(_ => Await.result(actorSystem.terminate(), Duration.Inf))(executor) // and shutdown when done
}
EDIT: Or in using the low-level akka-http API, how could I send specific messages to a sink from a specific route handler?
def main(args: Array[String]): Unit = {
implicit val actorSystem = ActorSystem("my-akka-http-test")
val executor = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()(actorSystem)
// I would like to send elements to this sink in response to HTTP GET operations.
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val requestHandler: HttpRequest => HttpResponse = {
case HttpRequest(GET, Uri.Path("/"), _, _, _) =>
HttpResponse(entity = HttpEntity(
ContentTypes.`text/html(UTF-8)`,
"<html><body>Hello world!</body></html>"))
case HttpRequest(GET, Uri.Path("/ping"), _, _, _) =>
HttpResponse(entity = "PONG!")
case HttpRequest(GET, Uri.Path("/crash"), _, _, _) =>
sys.error("BOOM!")
case r: HttpRequest =>
r.discardEntityBytes() // important to drain incoming HTTP Entity stream
HttpResponse(404, entity = "Unknown resource!")
}
val serverSource = Http().bind(interface = "localhost", port = 8080)
val bindingFuture: Future[Http.ServerBinding] =
serverSource.to(Sink.foreach { connection =>
println("Accepted new connection from " + connection.remoteAddress)
connection handleWithSyncHandler requestHandler
// this is equivalent to
// connection handleWith { Flow[HttpRequest] map requestHandler }
}).run()
println("Server online at http://localhost:8080/")
println("Press RETURN to stop...")
scala.io.StdIn.readLine()
bindingFuture
.flatMap(_.unbind())(executor) // trigger unbinding from the port
.onComplete(_ => Await.result(actorSystem.terminate(), Duration.Inf))(executor) // and shutdown when done
}
IF you want to send the whole HttpRequest to a sink of yours, I'd say the simplest way is to use the alsoTo combinator. The result would be something along the lines of
val mySink: Sink[HttpRequest, NotUsed] = ???
val handlerFlow = Flow[HttpRequest].alsoTo(mySink).via(RouteResult.route2HandlerFlow(route))
val bindingFuture = Http().bindAndHandle(handlerFlow, "localhost", 8080)
FYI: alsoTo in fact hides a Broadcast stage.
IF instead you need to selectively send a message to a Sink from a specific subroute, you have no other choice but to materialize a new flow for each incoming request. See example below
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val route: akka.http.scaladsl.server.Route =
path("hello" / Segment) { p =>
get {
(extract(_.request) & extractMaterializer) { (req, mat) ⇒
Source.single(req).runWith(sink)(mat)
complete {
s"<h1>Say hello to akka-http. p=$p</h1>"
}
}
}
}
Also, keep in mind you can always ditch the high-level DSL completely, and model you whole route using the lower-level streams DSL. This will result in more verbose code - but will give you full control of your stream materialization.
EDIT: example below
val sink: Sink[Any, Future[Done]] = Sink.foreach(println)
val handlerFlow =
Flow.fromGraph(GraphDSL.create() { implicit b =>
import GraphDSL.Implicits._
val partition = b.add(Partition[HttpRequest](2, {
case HttpRequest(GET, Uri.Path("/"), _, _, _) ⇒ 0
case _ ⇒ 1
}))
val merge = b.add(Merge[HttpResponse](2))
val happyPath = Flow[HttpRequest].map{ req ⇒
HttpResponse(entity = HttpEntity(
ContentTypes.`text/html(UTF-8)`,
"<html><body>Hello world!</body></html>"))
}
val unhappyPath = Flow[HttpRequest].map{
case HttpRequest(GET, Uri.Path("/ping"), _, _, _) =>
HttpResponse(entity = "PONG!")
case HttpRequest(GET, Uri.Path("/crash"), _, _, _) =>
sys.error("BOOM!")
case r: HttpRequest =>
r.discardEntityBytes() // important to drain incoming HTTP Entity stream
HttpResponse(404, entity = "Unknown resource!")
}
partition.out(0).alsoTo(sink) ~> happyPath ~> merge
partition.out(1) ~> unhappyPath ~> merge
FlowShape(partition.in, merge.out)
})
val bindingFuture = Http().bindAndHandle(handlerFlow, "localhost", 8080)
This is the solution I used that seems ideal. Akka Http seems like it's designed so that your routes are simple HttpRequest->HttpResponse flows and don't involve any extra branches.
Rather than build everything into a single Akka stream graph, I have a separate QueueSource->Sink graph, and the normal Akka Http HttpRequest->HttpResponse flow just adds elements to the source queue as needed.
object HttpWithSinkTest {
def buildQueueSourceGraph(): RunnableGraph[(SourceQueueWithComplete[String], Future[Done])] = {
val annotateMessage: Flow[String, String, NotUsed] = Flow.fromFunction[String, String](s => s"got message from queue: $s")
val sourceQueue = Source.queue[String](100, OverflowStrategy.dropNew)
val sink: Sink[String, Future[Done]] = Sink.foreach(println)
val annotatedSink = annotateMessage.toMat(sink)(Keep.right)
val queueGraph = sourceQueue.toMat(annotatedSink)(Keep.both)
queueGraph
}
def buildHttpFlow(queue: SourceQueueWithComplete[String],
actorSystem: ActorSystem, materializer: ActorMaterializer): Flow[HttpRequest, HttpResponse, NotUsed] = {
implicit val actorSystemI = actorSystem
implicit val materializerI = materializer
val route: akka.http.scaladsl.server.Route =
path("hello" / Segment) { p =>
get {
complete {
queue.offer(s"got http event p=$p")
s"<h1>Say hello to akka-http. p=$p</h1>"
}
}
}
val routeFlow = RouteResult.route2HandlerFlow(route)
routeFlow
}
def main(args: Array[String]): Unit = {
val actorSystem = ActorSystem("my-akka-http-test")
val executor = actorSystem.dispatcher
implicit val materializer = ActorMaterializer()(actorSystem)
val (queue, _) = buildQueueSourceGraph().run()(materializer)
val httpFlow = buildHttpFlow(queue, actorSystem, materializer)
val httpExt: akka.http.scaladsl.HttpExt = Http(actorSystem)
val bindingFuture = httpExt.bindAndHandle(httpFlow, "localhost", 8080)
println("Server online at http://localhost:8080/")
println("Press RETURN to stop...")
scala.io.StdIn.readLine()
println("Shutting down...")
val serverBinding = Await.result(bindingFuture, Duration.Inf)
Await.result(serverBinding.unbind(), Duration.Inf)
Await.result(actorSystem.terminate(), Duration.Inf)
println("Done. Exiting")
}
}
Im very new to akka and futures in scala. Im using spray to get the output of a URL and return a Future[String] to another object. Here is the object that is making the HTTP request.
object ActionsService {
private implicit val formats = DefaultFormats
implicit val system = ActorSystem()
import system.dispatcher
val pipeline = sendReceive ~> unmarshal[String]
def getActions: Future[String] ={
val out = getOutput("http://www.google.com")
out
}
def getOutput(url: String): Future[String] ={
val response = pipeline (Get (url) )
response
}
def shutdown(code: Int): Unit = {
IO(Http).ask(Http.CloseAll)(1.second).await
system.shutdown()
}
}
And here is the main method in the other object where I am trying to shutdown the actor system.
import ExecutionContext.Implicits.global
def main(args: Array[String]) {
val test = ActionsService.getActions
test.onComplete {
case Success(x) => println(x)
case Failure(y) => println(y)
}
ActionsService.system.shutdown()
For some reason the akka system wont shutdown. Ive also tried to shut down the akka system using my shutdown method but I get an Exception in thread "main" akka.pattern.AskTimeoutException: Timed out error (which also occurs in the main method when the shutdown method is called).
akka version is 2.2.3
You didn't specify version of Akka you're using but in last version you need to call
system.terminate()
instead and wait for termination like this (ass an example)
Await.ready(system.whenTerminated, Duration.Inf)
The answer above did not end up working for me so I had to block the thread by adding a sleep period for the main method before the shutdown method is called.
def main(args: Array[String]) {
val test = ActionsService.getActions
Thread.sleep(700)
test.onComplete {
case Success(x) => println(x)
case Failure(y) => println(y)
}
CorporateActionsService.system.shutdown()
}
I'm using spray-client to generate http requests to my server in e2e tests. I also use specs2 to test for the desired response from the server. And everything works fine.
I've built some custom specs2 matchers to simplify my test code. My test looks like this:
val response = get(<server_endpoint_url>)
response must beSuccessfulWith(content = expected_data)
I have a trait that somewhat simplifies the usage of spray in the test itself:
trait SprayTestClientSupport {
implicit val system = ActorSystem()
import system.dispatcher // execution context for futures
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
def get(url: String): Future[HttpResponse] = pipeline(Get(url))
}
I also have a trait where I define the custom matchers I use in the test:
trait SprayTestClientSupport extends ShouldMatchers with SprayJsonSupport with DefaultJsonProtocol {
def beSuccessfulWith(content: Seq[Int]): Matcher[Future[HttpResponse]] =
beSuccessful and haveBodyWith(content)
def haveBodyWith(content: Seq[Int]): Matcher[Future[HttpResponse]] =
containTheSameElementsAs(content) ^^ { f: Future[HttpResponse] =>
Await.result(f, await).entity.as[Seq[Int]].right.get
}
def beSuccessful: Matcher[Future[HttpResponse]] =
===(StatusCode.int2StatusCode(200)) ^^ { f: Future[HttpResponse] =>
Await.result(f, await).status
}
}
My problem starts when I try to make the matchers more general and support any Scala type for instance. I define something like this:
def haveBodyWith[T: TypeTag](content: T): Matcher[Future[HttpResponse]] =
===(content) ^^ { f: Future[HttpResponse] =>
Await.result(f, await).entity.as[T].right.get
}
But then I get the following error message:
Error:(49, 86) could not find implicit value for parameter unmarshaller: spray.httpx.unmarshalling.Unmarshaller[T]
===(content) ^^ { (f: Future[HttpResponse]) => { Await.result(f, await).entity.as[T].right.get } }
Is there anything simple that I'm missing?
Thanks!
P.S.
I use the following spray versions:
spray-client_2.10 -> 1.3.3
spray-can_2.10 -> 1.3.3
spray-http_2.10 -> 1.3.3
spray-httpx_2.10 -> 1.3.3
spray-util_2.10 -> 1.3.3
spray-json_2.10 -> 1.3.2
You need to add a constraint to your T parameter
def haveBodyWith[T: TypeTag : Unmarshaller](content: T): Matcher[Future[HttpResponse]] =
===(content) ^^ { f: Future[HttpResponse] =>
Await.result(f, await).entity.as[T].right.get
}
I have a client that makes a REST call to a server that exposes REST API's. Here is what I have so far implemented:
override def getData[T](reqURL: String, requestParam: Option[Seq[(String, String)]])(responseHandler: T => Unit) = {
val onSuccess: PartialFunction[String, Unit] = {
case s => responseHandler(s.asInstanceOf[T])
}
val onFailure: PartialFunction[Throwable, Unit] = {
case e => e.printStackTrace
}
val request = requestParam match {
case Some(params) => url(reqURL).as_!("user", "pass") <<? params
case None => url(reqURL).as_!("user", "pass")
}
Http(request OK as.String).onSuccess(onSuccess).onFailure(onFailure)
}
It works perfectly fine, but what I noticed is that the HTTP connection is open even after the call is complete. How can I close this connection? I might want to do it after the onSuccess or onFailure call has happened. Any suggestions?