I'm using http4s BlazeServer 0.21, how can I graceful shutdown? I want to reject all upcoming requests, and keep process unfinished requests and response back, within a hard shutdown time.
I tried starting server with serveWhile and set a shutdownHook SignallingRef. The server stream & middleware defer as expected (so our metrics & log middleware still log this response)
//serverStream
for {
signal <- fs2.Stream.eval(SignallingRef[F, Boolean](false))
exitCode <- fs2.Stream.eval(Ref[F].of(ExitCode.Success))
_ <- fs2.Stream.eval(shutdown(signal))
server <- BlazeServerBuilder[F]
.bindHttp(8080, "0.0.0.0")
.withHttpApp(httpApp)
.serveWhile(signal, exitCode)
} yield server
def shutdown[F[_]: Effect](interrupter: SignallingRef[F, Boolean]): F[Unit] = {
LiftIO[F].liftIO(IO {
sys.addShutdownHook {
...
interrupter.set(true)
}
})
}
object Server extends IOApp {
def run(args: List[String]): IO[ExitCode] =
serverStream[IO].compile.drain.as(ExitCode.Success)
}
but the http server doesn't work as I expect, seems like http4s's internal ServerChannel has its own shutdownHook and cancel all the responses already.
any suggestion/workaround? or maybe just a way to hold and don't kill requests for x seconds is also appreciated.
The server is hooked for SIGTERM as a matter of convenience.
As a convenience, cats-effect provides an cats.effect.IOApp trait with
an abstract run method that returns a IO[ExitCode]. An IOApp runs the
process and adds a JVM shutdown hook to interrupt the infinite process
and gracefully shut down your server when a SIGTERM is received.
And if you want to shutdown using an URL e.g. http://localhost:8080/ops/shutdown/true, it works too.
The code is pretty straight forward,
class SysOpsEndpoints[F[_]: Sync](signal: SignallingRef[F, Boolean]) extends Http4sDsl[F] {
private def shutdown: HttpRoutes[F] =
HttpRoutes.of[F] {
case GET -> Root / "shutdown" / shutdown =>
for {
_ <- signal.set(Try(shutdown.toBoolean).getOrElse(false))
result <- Ok(s"Shutdown: $shutdown")
} yield result
}
}
object SysOpsEndpoints {
def endpoints[F[_]: Sync](signal: SignallingRef[F, Boolean]): HttpRoutes[F] =
new SysOpsEndpoints(signal).shutdown
}
and the setup for the server for-comprehension is similar to yours,
for {
signal <- fs2.Stream.eval(SignallingRef[F, Boolean](false))
exitCode <- fs2.Stream.eval(Ref[F].of(ExitCode.Success))
httpApp = Router(
"/ops" -> SysOpsEndpoints.endpoints(signal)
).orNotFound
server <- BlazeServerBuilder[F](serverEc)
.bindHttp(8080, "0.0.0.0")
.withHttpApp(httpApp)
.serveWhile(signal, exitCode)
} yield server
Related
I am having issues testing my first ZIO+HTTP4S application. The test hangs and does not finish.
The code for my App (simplified) is
object Main extends App {
def server: ZIO[AppEnvironment, Throwable, Unit] =
for {
(...)
fullApp <- ZIO.runtime[AppEnvironment].flatMap { implicit rts =>
BlazeServerBuilder[AppTask](ec)
.bindHttp(api.port, api.endpoint)
.withHttpApp(CORS(httpApp))
.serve
.compile[AppTask, AppTask, CatsExitCode]
.drain
}
} yield fullApp
override def run(args: List[String]): ZIO[ZEnv, Nothing, ExitCode] = {
server.provideSomeLayer[ZEnv](appEnvironment).tapError(err => putStrLn(s"Execution failed with: $err")).exitCode
}
}
This is my test code. Note that it is basically copypasted from THIS OTHER STACKOVERFLOW QUESTION
object ApiTest extends DefaultRunnableSpec {
val ec: ExecutionContext = ExecutionContext.global
def httpServer = Main.run(List()).forkManaged.toLayer
val clockDuration = ofSeconds(1)
val blocker = Blocker.liftExecutionContext(ec)
//did the httpserver start listening on 8080?
private def isLocalPortInUse(port: Int): ZIO[Clock, Throwable, Unit] = {
IO.effect {
println("checking for local port in use")
new Socket("0.0.0.0", port).close()
}
.retry(Schedule.linear(clockDuration) && Schedule.recurs(10))
}
override def spec: ZSpec[Environment, Failure] =
suite("MainTest")(
testM("Health check") {
for {
_ <- TestClock.adjust(clockDuration).fork
_ = println("1")
_ <- isLocalPortInUse(8080)
_ = println("2")
client <- Task(JavaNetClientBuilder[Task](blocker).create)
_ = println("3")
response <- client.expect[String]("http://localhost:8080/healthcheck")
_ = println("4")
} yield assert(response)(equalTo(""))
}
).provideCustomLayerShared(httpServer)
}
The problem is that as soon as the server starts, the test stops running and are not executed. The output is
1
checking for local port in use
checking for local port in use
<server init message>
In Suite "MainTest", test "Health check" has taken more than 1 m to execute. If this is not expected, consider using TestAspect.timeout to timeout runaway tests for faster diagnostics.
So as you can see, the tests run OK until the server starts and then does not continue the execution.
Also, how could I perform a POST call instead of a GET one? I'm a bit lost in the HTTP4S/ZIO ecosystem with http4sClients, ZHTTP, BlazeClientBuilders and the like. What would be the easiest way of doing a POST call to my server in a test like the previous one?
Cheers!
edit: I’ve checked that the server works fine while hanging in here, I can do CURL calls from a terminal and it responds correctly. So it seems clear that the problem is that as soon as the server is up it stays in the front, not background, and the tests do not have the chance to finish the execution.
You are advancing the clock by 1 second but your application might require more time to run. Also, your particular test will require infinite time to run because, while unit tests are instantaneous in ZIO, integration tests are not.
Advancing the time of a unit test by 1 second requires theoretically 0 seconds. This might not be enough for the port to become free.
Since you are trying to create an integration test, you should use a real Clock and not the one provided by the test kit.
I have the following code:
class ApiRoutes2[F[_]](implicit F: ConcurrentEffect[F]) extends Http4sDsl[F] {
var queue = Queue.bounded[F, String](100)
def createService(queue: Queue[F, String]): F[Unit] = ???
val service: HttpRoutes[F] = HttpRoutes.of[F] {
case PUT -> Root / "services" =>
val toClientF: F[Stream[F, WebSocketFrame]] = queue.map(_.dequeue.map(t => Text(t)))
val fromClient: Pipe[F, WebSocketFrame, Unit] = _.evalMap {
case Text(t, _) => F.delay(println(t))
case f => F.delay(println(s"Unknown type: $f"))
}
// How to "spawn" createService?
toClientF.flatMap { toClient =>
WebSocketBuilder[F].build(toClient, fromClient)
}
}
}
createService is a function which creates a new service. Creating a new service is a very complicated process, it envolves triggering CI pipelines, waiting for them to finish and then trigger more CI pipelines in the same fashion. The queue it receives will be used to report back to the browser the current operations being performed.
I wanna concurrently "spawn" the createService and let it run until it finishes. However at the same time I want to immediately return the WebSocket to the client. Aka I cannot block while "spawning" createService.
I'm stuck. I can only think of using shift but that would mean the next line in the for comprehension would block waiting for createService to finish only to then return the websocket to the client.
Is my approach wrong? What am I doing wrong?
Since F is an instance of ConcurrentEffect, you also have a Concurrent instance.
You can therefore use Concurrent[F].start which returns a Fiber to the running operation (you can just ignore the Fibre if you don't need to cancel/ensure completion though).
val service: HttpRoutes[F] = HttpRoutes.of[F] {
case PUT -> Root / "services" =>
val toClientF: F[Stream[F, WebSocketFrame]] = queue.map(_.dequeue.map(t => Text(t)))
val fromClient: Pipe[F, WebSocketFrame, Unit] = _.evalMap {
case Text(t, _) => F.delay(println(t))
case f => F.delay(println(s"Unknown type: $f"))
}
for {
toClient <- toClientF
_ <- Concurrent[F].start(createService)
websocket <- WebSocketBuilder[F].build(toClient, fromClient)
} yield websocket
}
I want to implement a high-throughput server that accepts multiple clients. Every request should query a database, so I need some kind of async behavior.
I followed the ROUTER-to-REQ pattern from documentation + Futures, so I ended with this "architecture":
trait ZmqProtocol extends Protocol {
private val pool = Executors.newCachedThreadPool()
private implicit val ec: ExecutionContextExecutor = ExecutionContext.fromExecutor(pool)
val context: ZMQ.Context = ZMQ.context(1)
val socket: ZMQ.Socket = context.socket(ZMQ.ROUTER)
socket.bind("tcp://*:5555")
override def receiveMessages(): String = {
while (true) {
val address = socket.recv(0)
val empty = socket.recv(0)
val request = socket.recv(0)
Future {
val message = new String(request)
getResponseFromDb(message)
} onComplete {
case Success(response) =>
// Send reply back to client
socket.send(address, ZMQ.SNDMORE)
socket.send("".getBytes, ZMQ.SNDMORE)
socket.send(response.getBytes(), 0)
case Failure(ex) => println(ex)
}
}
"DONE"
}
}
I understand this won't work because I'm sharing socket in Future so I need a better model. I know the ZeroMQ sockets are fast and creating several worker threads would be enough on input side, but if the bottleneck is on the database side and if I need to do some other work while waiting for DB, I presume all my threads would soon be exhausted.
Would it be too much of an overhead if I create new socket and bind on ROUTER in every Future or is there some better solution?
Also, for Scala developers: is there a way to force onComplete being executed on main thread (I suppose it would solve the issue)? Thanks!
I have an HTTP Connection Pool that hangs after a couple of hours of running:
private def createHttpPool(host: String): SourceQueue[(HttpRequest, Promise[HttpResponse])] = {
val pool = Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]](host)
Source.queue[(HttpRequest, Promise[HttpResponse])](config.poolBuffer, OverflowStrategy.dropNew)
.via(pool).toMat(Sink.foreach {
case ((Success(res), p)) => p.success(res)
case ((Failure(e), p)) => p.failure(e)
})(Keep.left).run
}
I enqueue items with:
private def enqueue(uri: Uri): Future[HttpResponse] = {
val promise = Promise[HttpResponse]
val request = HttpRequest(uri = uri) -> promise
queue.offer(request).flatMap {
case Enqueued => promise.future
case _ => Future.failed(ConnectionPoolDroppedRequest)
}
}
And resolve the response like this:
private def request(uri: Uri): Future[HttpResponse] = {
def retry = {
Thread.sleep(config.dispatcherRetryInterval)
logger.info(s"retrying")
request(uri)
}
logger.info("req-start")
for {
response <- enqueue(uri)
_ = logger.info("req-end")
finalResponse <- response.status match {
case TooManyRequests => retry
case OK => Future.successful(response)
case _ => response.entity.toStrict(10.seconds).map(s => throw Error(s.toString, uri.toString))
}
} yield finalResponse
}
The result of this function is then always transformed if the Future is successful:
def get(uri: Uri): Future[Try[JValue]] = {
for {
response <- request(uri)
json <- Unmarshal(response.entity).to[Try[JValue]]
} yield json
}
Everything works fine for a while and then all I see in the logs are req-start and no req-end.
My akka configuration is like this:
akka {
actor.deployment.default {
dispatcher = "my-dispatcher"
}
}
my-dispatcher {
type = Dispatcher
executor = "fork-join-executor"
fork-join-executor {
parallelism-min = 256
parallelism-factor = 128.0
parallelism-max = 1024
}
}
akka.http {
host-connection-pool {
max-connections = 512
max-retries = 5
max-open-requests = 16384
pipelining-limit = 1
}
}
I'm not sure if this is a configuration problem or a code problem. I have my parallelism and connection numbers so high because without it I get very poor req/s rate (I want to request as fast possible - I have other rate limiting code to protect the server).
You are not consuming the entity of the responses you get back from the server. Citing the docs below:
Consuming (or discarding) the Entity of a request is mandatory! If
accidentally left neither consumed or discarded Akka HTTP will assume
the incoming data should remain back-pressured, and will stall the
incoming data via TCP back-pressure mechanisms. A client should
consume the Entity regardless of the status of the HttpResponse.
The entity comes in the form of a Source[ByteString, _] which needs to be run to avoid resource starvation.
If you don't need to read the entity, the simplest way to consume the entity bytes is to discard them, by using
res.discardEntityBytes()
(you can attach a callback by adding - e.g. - .future().map(...)).
This page in the docs describes all the alternatives to this, including how to read the bytes if needed.
--- EDIT
After more code/info was provided, it is clear that the resource consumption is not the problem. There is another big red flag in this implementation, namely the Thread.sleep in the retry method.
This is a blocking call that is very likely to starve the threading infrastructure of your underlying actor system.
A full blown explanation of why this is dangerous was provided in the docs.
Try changing that and using akka.pattern.after (docs). Example below:
def retry = akka.pattern.after(200 millis, using = system.scheduler)(request(uri))
I want to make a request to a server asynchronously involving actors. Say I have 2 actors:
class SessionRetriever extends Actor {
import SessionRetriever._
def receiver = {
Get =>
val s = getSessionIdFromServer() // 1
sender ! Result(s) // 2
}
def getSessionIdFromServer(): String = { ... } // 3
}
object SessionRetriever {
object Get
object Result(s: String)
}
And
class RequestSender extends Actor {
val sActor = context actorOf Props[SessionRetriever]
def receiver = {
// get session id
val sesId = sActor ! SessionRetriever.Get
val res = sendRequestToServer(sesId)
logToFile(res)
context shutdown sActor
}
def sendRequestToServer(sessionId: String): String = { .... }
}
My questions:
val s = getSessionIdFromServer() // 1
sender ! Result(s) // 2
1) getSessionIdFromServer() does a synchronous request to the server. I think it would be much better make a request asynchronous, correct? So it will return Future[String] instead of a plain String.
2) How do I make asynchronous: by using an AsyncHttpClient (if I recall correctly its name) or by wrapping its synchronous body into Future { } ?
3) Should I use blocking { } block ? If yes, then where exactly: inside its body or here val s = blocking { getSessionIdFromServer() } ?
P.S. I'd like not to use async { } and await { } at this point because they are quite high level functions and after all they are build on top on Futures.
you might try this non-blocking way
def receive = {
Get =>
//assume getSessionIdFromServer() run aysnchronize
val f: Future[String] = getSessionIdFromServer()
val client = sender //keep it local to use when future back
f onComplete {
case Success(rep) => client ! Result(rep)
case Failure(ex) => client ! Failed(ex)
}
}
1) If getSessionIdFromServer() is blocking then you should execute it asynchronously from your receive function, otherwise your actor will block each time it receives a new request and will always wait until it receives a new session before processing the next request.
2) Using a Future will "move" the blocking operation to a different thread. So, your actor will not be blocked and will be able to keep processing incoming requests - that's good -, however you are still blocking a thread - not so great. Using the AsyncHttpClient is a good idea. You can explore other non-blocking httpClient, like PlayWebService.
3) I am not quite familiar with blocking so not sure I should advise anything here. From what I understand, it will tell the thread pool that the operation is blocking and that it should spawn a temporary new thread to handle it - this avoids having all your workers being blocked. Again, if you do that your actor will not blocked, but you are still blocking a thread while getting the session from the server.
To summarize: just use an async http client in getSessionIdFromServer if it is possible. Otherwise, use either Future{} or blocking.
To do an asynchronous call with AsyncHttpClient you could deal with the java Future via a scala Promise.
import scala.concurrent.Future
import com.ning.http.client.AsyncHttpClient
import scala.concurrent.Promise
import java.util.concurrent.Executor
object WebClient {
private val client = new AsyncHttpClient
case class BadStatus(status: Int) extends RuntimeException
def get(url: String)(implicit exec: Executor): Future[String] = {
val f = client.prepareGet(url).execute();
val p = Promise[String]()
f.addListener(new Runnable {
def run = {
val response = f.get
if (response.getStatusCode / 100 < 4)
p.success(response.getResponseBodyExcerpt(131072))
else p.failure(BadStatus(response.getStatusCode))
}
}, exec)
p.future
}
def shutdown(): Unit = client.close()
}
object WebClientTest extends App {
import scala.concurrent.ExecutionContext.Implicits.global
WebClient get "http://www.google.com/" map println foreach (_ => WebClient.shutdown())
}
And then deal with the future completion via a callback.
Credit for the code to the awesome reactive programming course at Coursera.