I'm writing an Interactive Broker API using Scala and Akka actors.
I have a Client actor that connects to the server and communicate with the IO manager to send requests and receive responses from TWS. The connection works fine and I'm able to send a request and get the response.
Then I receive automatically a PeerClosed message from the IO manager after 1 minute. I would like that the connection stays open unless I explicitly close it. I tried to set keepOpenOnPeerClosed = true but it changes nothing.
Here is the Actor:
class Client(remote: InetSocketAddress, clientId: Int, extraAuth: Boolean, onConnected: Session => Unit, listener: EWrapper) extends Actor {
final val ClientVersion: Int = 63
final val ServerVersion: Int = 38
final val MinServerVerLinking: Int = 70
import Tcp._
import context.system
IO(Tcp) ! Connect(remote)
def receive = {
case CommandFailed(_: Connect) =>
print("connect failed")
context stop self
case c#Connected(remote, local) => {
val connection = sender()
connection ! Register(self, keepOpenOnPeerClosed = true)
context become connected(connection,1)
val clientVersionBytes = ByteString.fromArray(String.valueOf(ClientVersion).getBytes() ++ Array[Byte](0.toByte))
println("Sending Client Version " + clientVersionBytes)
sender() ! Write(clientVersionBytes)
}
}
def connected(connection: ActorRef, serverVersion: Int): Receive = {
case request: Request =>
print("Send request " + request)
connection ! Write(ByteString(request.toBytes(serverVersion)))
case CommandFailed(w: Write) =>
connection ! Close
print("write failed")
case Received(data) => {
println(data)
implicit val is = new DataInputStream(new ByteArrayInputStream(data.toArray))
EventDispatcher.consumers.get(readInt()) match {
case Some(consumer) => {
consumer.consume(listener, serverVersion)
}
case None => {
listener.error(EClientErrors.NoValidId, EClientErrors.UnknownId.code, EClientErrors.UnknownId.msg)
}
}
}
case _ : ConnectionClosed => context stop self
}
I don't have the same behaviour if I connect using IBJts API (using a standard Java Socket)
Have you tried it with the keep alive option?
sender ! Tcp.SO.KeepAlive(on = true)
Related
i am using akka http one of my routes is interacting with an external service via akka http client side api and the httpRequest is continuously running i am unable to make it work
here is my use case -> i am interacting with a janus server and doing a long poll get request as soon as the server responded back with an 'keepAlive' or an "event" i am requesting again and so on the server keeps on responding
all of this is happening inside an actor and i have an akka htttp route which is intiailising the first request
here is my code
final case class CreateLongPollRequest(sessionId:BigInt)
class LongPollRequestActor (config: Config) extends Actor {
def receive = {
case CreateLongPollRequest(sessionId) =>
senderRef = Some(sender())
val uri: String = "localhost:8080/" + sessionId
val request = HttpRequest(HttpMethods.GET, uri)
val responseFuture = Http(context.system).singleRequest(request)
responseFuture
.onComplete {
case Success(res)
Unmarshal(res.entity.toStrict(40 seconds)).value.map { result =>
val responseStr = result.data.utf8String
log.info("Actor LongPollRequestActor: long poll responseStr {}",responseStr)
senderRef match {
case Some(ref) =>
ref ! responseStr
case None => log.info("Actor LongPollRequestActor: sender ref is null")
}
}
case Failure(e) =>log.error(e)
}
}
}
final case class JanusLongPollRequest(sessionId: BigInt)
class JanusManagerActor(childMaker: List[ActorRefFactory => ActorRef]) extends Actor {
var senderRef: Option[akka.actor.ActorRef] = None
val longPollRequestActor = childMaker(1)(context)
def receive: PartialFunction[Any, Unit] = {
case JanusLongPollRequest(sessionId)=>
senderRef = Some(sender)
keepAlive(sessionId,senderRef)
}
def keepAlive(sessionId:BigInt,sender__Ref: Option[ActorRef]):Unit= {
val senderRef = sender__Ref
val future = ask(longPollRequestActor, CreateLongPollRequest(sessionId)).mapTo[String] //.pipeTo(sender)
if (janus.equals("keepalive")) {
val janusRequestResponse = Future {
JanusSessionRequestResponse(janus = janus)
}
senderRef match {
case Some(sender_ref) =>
janusRequestResponse.pipeTo(sender_ref)
}
keepAlive(sessionId,senderRef)
}
else if (janus.equals("event")) {
//some fetching of values from server
val janusLongPollRequestResponse = Future {
JanusLongPollRequestResponse(janus = janus,sender=sender, transaction=transaction,pluginData=Some(pluginData))
}
senderRef match {
case Some(sender_ref) =>
janusLongPollRequestResponse.pipeTo(sender_ref)
}
keepAlive(sessionId,senderRef)
}
def createLongPollRequest: server.Route =
path("create-long-poll-request") {
post {
entity(as[JsValue]) {
json =>
val sessionID = json.asJsObject.fields("sessionID").convertTo[String]
val future = ask(janusManagerActor, JanusLongPollRequest(sessionID)).mapTo[JanusSessionRequestResponse]
onComplete(future) {
case Success(sessionDetails) =>
log.info("janus long poll request created")
val jsonResponse = JsObject("longpollDetails" -> sessionDetails.toJson)
complete(OK, routeResponseMessage.getResponse(StatusCodes.OK.intValue, ServerMessages.JANUS_SESSION_CREATED, jsonResponse))
case Failure(ex) =>
failWith(ex)
}
}
}
now the above route createLongPollRequest worked fine for the first time I can see the response and for the next attempts i am getting a dead letter as follows
[INFO] [akkaDeadLetter][07/30/2021 12:13:53.587] [demo-Janus-ActorSystem-akka.actor.default-dispatcher-6] [akka://demo-Janus-ActorSystem/deadLetters] Message [com.ifkaar.lufz.janus.models.janus.JanusSessionRequestResponse] from Actor[akka://demo-Janus-ActorSystem/user/ActorManager/ManagerActor#-721316187] to Actor[akka://demo-Janus-ActorSystem/deadLetters] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://demo-Janus-ActorSystem/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
probably this is causing the issue after the first iteration
responseFuture.pipeTo(sender()
IS there a way where i can get a response in my akkahttp route when ever my backend server responds?
The Actor should only reply once to the CreateLongPollRequest and it should only do this when it has valid data. If the poll fails the Actor should just issue another poll request.
It is difficult to give more help without the details of the Actor.
I'm trying to implement a reverse HTTP proxy with Spray/Akka, but runs into trouble. I found that under some circumstances, my proxy server will keep receivving data from upstream server even after the client has disconnected.
Here's how I implement my Spray proxy directive (just a little modification to bthuillier's implementation):
trait ProxyDirectives {
private def sending(f: RequestContext ⇒ HttpRequest)(implicit system: ActorSystem): Route = {
val transport = IO(Http)(system)
ctx ⇒ transport.tell(f(ctx), ctx.responder)
}
/**
* Re-shape the original request, to match the destination server.
*/
private def reShapeRequest(req: HttpRequest, uri: Uri): HttpRequest = {
req.copy(
uri = uri,
headers = req.headers.map {
case x: HttpHeaders.Host => HttpHeaders.Host(uri.authority.host.address, uri.authority.port)
case x => x
}
)
}
/**
* proxy the request to the specified uri
*
*/
def proxyTo(uri: Uri)(implicit system: ActorSystem): Route = {
sending(ctx => reShapeRequest(ctx.request, uri))
}
}
This reverse proxy will work well if I put one proxy layer between the client and the server (that is, client <-> proxyTo <-> server), but it will have trouble if I put two layers between the client and the server. For example, if I've got the following simple Python HTTP server:
import socket
from threading import Thread, Semaphore
import time
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from SocketServer import ThreadingMixIn
class MyHTTPHandler(BaseHTTPRequestHandler):
protocol_version = 'HTTP/1.1'
def do_GET(self):
self.send_response(200)
self.send_header('Transfer-Encoding', 'chunked')
self.end_headers()
for i in range(100):
data = ('%s\n' % i).encode('utf-8')
self.wfile.write(hex(len(data))[2:].encode('utf-8'))
self.wfile.write(b'\r\n')
self.wfile.write(data)
self.wfile.write(b'\r\n')
time.sleep(1)
self.wfile.write(b'0\r\n\r\n')
class MyServer(ThreadingMixIn, HTTPServer):
def server_bind(self):
HTTPServer.server_bind(self)
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
def server_close(self):
HTTPServer.server_close(self)
if __name__ == '__main__':
server = MyServer(('127.0.0.1', 8080), MyHTTPHandler)
server.serve_forever()
Which basically does nothing but open a chunked response (for long-term running, so that we can exam the issues). And if I chain two layers of proxies in the following way:
class TestActor(val target: String)(implicit val system: ActorSystem) extends Actor
with HttpService
with ProxyDirectives
{
// we use the enclosing ActorContext's or ActorSystem's dispatcher for our Futures and Scheduler
implicit private def executionContext = actorRefFactory.dispatcher
// the HttpService trait defines only one abstract member, which
// connects the services environment to the enclosing actor or test
def actorRefFactory = context
val serviceRoute: Route = {
get {
proxyTo(target)
}
}
// runs the service routes.
def receive = runRoute(serviceRoute) orElse handleTimeouts
private def handleTimeouts: Receive = {
case Timedout(x: HttpRequest) =>
sender ! HttpResponse(StatusCodes.InternalServerError, "Request timed out.")
}
}
object DebugMain extends App {
val actorName = "TestActor"
implicit val system = ActorSystem(actorName)
// create and start our service actor
val service = system.actorOf(
Props { new TestActor("http://127.0.0.1:8080") },
s"${actorName}Service"
)
val service2 = system.actorOf(
Props { new TestActor("http://127.0.0.1:8081") },
s"${actorName}2Service"
)
IO(Http) ! Http.Bind(service, "::0", port = 8081)
IO(Http) ! Http.Bind(service2, "::0", port = 8082)
}
Use curl http://localhost:8082 to connect to the proxy server, and you will see the Akka system keeps transferring data even after curl has been killed (you may turn on the logs of DEBUG level to see details).
How can I deal with this problem? Thanks.
Well, it turns out to be a very complex problem, while my solution takes nearly 100 lines of codes.
Actually, the problem does not only exist when I'm stacking two layers of proxies. When I'm using one layer proxy, the problem does exist, but no log is printed, so I've not aware of this problem before.
The key problem is that while we use IO(Http) ! HttpRequest, it is actually a host-level API from spray-can. The connections of host-level APIs are managed by Spray HttpManager, which is not accessible by our code. Thus we can do nothing with that connection, unless we send a Http.CloseAll to IO(Http), which will cause all the upstream connections to be closed.
(If anyone knows how to get the connection from HttpManager, please tell me).
We have to use connection-level APIs from spray-can to serve for this situation. So I've come up with something like this:
/**
* Proxy to upstream server, where the server response may be a long connection.
*
* #param uri Target URI, where to proxy to.
* #param system Akka actor system.
*/
def proxyToLongConnection(uri: Uri)(implicit system: ActorSystem): Route = {
val io = IO(Http)(system)
ctx => {
val request = reShapeRequest(ctx.request, uri)
// We've successfully opened a connection to upstream server, now start proxying data.
actorRefFactory.actorOf {
Props {
new Actor with ActorLogging {
private var upstream: ActorRef = null
private val upstreamClosed = new AtomicBoolean(false)
private val clientClosed = new AtomicBoolean(false)
private val contextStopped = new AtomicBoolean(false)
// Connect to the upstream server.
{
implicit val timeout = Timeout(FiniteDuration(10, TimeUnit.SECONDS))
io ! Http.Connect(
request.uri.authority.host.toString,
request.uri.effectivePort,
sslEncryption = request.uri.scheme == "https"
)
context.become(connecting)
}
def connecting: Receive = {
case _: Http.Connected =>
upstream = sender()
upstream ! request
context.unbecome() // Restore the context to [[receive]]
case Http.CommandFailed(Http.Connect(address, _, _, _, _)) =>
log.warning("Could not connect to {}", address)
complete(StatusCodes.GatewayTimeout)(ctx)
closeBothSide()
case x: Http.ConnectionClosed =>
closeBothSide()
}
override def receive: Receive = {
case x: HttpResponse =>
ctx.responder ! x.withAck(ContinueSend(0))
case x: ChunkedMessageEnd =>
ctx.responder ! x.withAck(ContinueSend(0))
case x: ContinueSend =>
closeBothSide()
case x: Failure =>
closeBothSide()
case x: Http.ConnectionClosed =>
closeBothSide()
case x =>
// Proxy everything else from server to the client.
ctx.responder ! x
}
private def closeBothSide(): Unit = {
if (upstream != null) {
if (!upstreamClosed.getAndSet(true)) {
upstream ! Http.Close
}
}
if (!clientClosed.getAndSet(true)) {
ctx.responder ! Http.Close
}
if (!contextStopped.getAndSet(true)) {
context.stop(self)
}
}
} // new Actor
} // Props
} // actorOf
} // (ctx: RequestContext) => Unit
}
The code is little long, and I doubt there should be some more clean and simple implementation (actually I'm not familiar with Akka). Nevertheless, this code works, so I put this solution here. You may post your solution to this problem freely, if you've found some better one.
I'm using scala-io in my akka actors, in my case I need to send request and wait for response, in official docs (http://doc.akka.io/docs/akka/snapshot/scala/io-tcp.html) I can see the answer is asynchronous.
How can I wait for response, can I use somehow ? (ask) pattern
class SocketClient(remoteAddress: InetSocketAddress, listener: ActorRef) extends Actor {
import Tcp._
import context.system
IO(Tcp) ! Connect(remoteAddress)
def receive = {
case CommandFailed(_: Connect) =>
listener ! ConnectFailure
context stop self
case Connected(remote, local) =>
listener ! ConnectSuccess
val connection = sender
connection ! Register(self)
context become {
case data: ByteString =>
connection ! Write(data)
case CommandFailed(w: Write) =>
Logger.error(s"Error during writing")
case Received(data) =>
listener ! data
case Disconnect =>
connection ! Close
case _: ConnectionClosed =>
Logger.error(s"Connection has been closed ${remoteAddress.getAddress}")
context stop self
}
}
}
Can I use something like:
connection ? Write(data)
Yes, but you should take in account the fact that ask-pattern allows you to receive only first reply from actor.
In your case it's connection, which may reply some additional or even unrelated objects (it depends on back-pressure/acknowledgement mode you choose. For example, if you use Write - you may receive the written (to the socket) object's acknowledge instead of response.
You can avoid it by:
using NoAck as an AckEvent (see http://doc.akka.io/docs/akka/snapshot/scala/io-tcp.html, Throttling Reads and Writes section).
use atomic requests/replies (no multi-part)
use one actor per protocol (per each "ping-pong" sequence)
In other words, ask pattern just creates own internal actor per message and make it a sender, so all replies (for this particular message) are going to this micro-actor. When it receives first reply - future (returned by ?) becomes completed - and internal actor destroyed (so other replies will be ignored).
Also, connection automatically replies to registered (by Register message) listener instead of sender - so you should create mediate actor:
class Asker(connection: ActorRef) extends Actor {
import Tcp._
connection ! Register(self);
def receive = {
case x =>
val parent = sender()
connection ! x
context become {case x => parent ! x; context.unbecome()}
}
}
trait TcpAskSupport {
self: Actor =>
def asker(connection: ActorRef) =
context.child(connection.path.name + "Asker")
.getOrElse(system.actorOf(Props(classOf[Asker], connection),
connection.path.name + "Asker"))
}
Usage example:
class Client extends Actor with TcpAskSupport {
import Tcp._
import context.system
IO(Tcp) ! Connect(new InetSocketAddress("61.91.16.168", 80))
implicit val timeout = Timeout(new FiniteDuration(5, SECONDS))
def receive = {
case CommandFailed(_: Connect) =>
println("connect failed")
context stop self
case c # Connected(remote, local) =>
println("Connected" + c)
val connection = sender()
asker(connection) ? Write(ByteString("GET /\n", "UTF-8"), NoAck) onComplete {
case x => println("Response" + x)
}
case x => println("[ERROR] Received" + x)
}
}
In my below test, I tried to simulate a timeout and then send a normal request. however, I got spray.can.Http$ConnectionException: Premature connection close (the server doesn't appear to support request pipelining)
class SprayCanTest extends ModuleTestKit("/SprayCanTest.conf") with FlatSpecLike with Matchers {
import system.dispatcher
var app = Actor.noSender
protected override def beforeAll(): Unit = {
super.beforeAll()
app = system.actorOf(Props(new MockServer))
}
override protected def afterAll(): Unit = {
system.stop(app)
super.afterAll()
}
"response time out" should "work" in {
val setup = Http.HostConnectorSetup("localhost", 9101, false)
connect(setup).onComplete {
case Success(conn) => {
conn ! HttpRequest(HttpMethods.GET, "/timeout")
}
}
expectMsgPF() {
case Status.Failure(t) =>
t shouldBe a[RequestTimeoutException]
}
}
"normal http response" should "work" in {
//Thread.sleep(5000)
val setup = Http.HostConnectorSetup("localhost", 9101, false)
connect(setup).onComplete {
case Success(conn) => {
conn ! HttpRequest(HttpMethods.GET, "/hello")
}
}
expectMsgPF() {
case HttpResponse(status, entity, _, _) =>
status should be(StatusCodes.OK)
entity should be(HttpEntity("Helloworld"))
}
}
def connect(setup: HostConnectorSetup)(implicit system: ActorSystem) = {
// for the actor 'asks'
import system.dispatcher
implicit val timeout: Timeout = Timeout(1 second)
(IO(Http) ? setup) map {
case Http.HostConnectorInfo(connector, _) => connector
}
}
class MockServer extends Actor {
//implicit val timeout: Timeout = 1.second
implicit val system = context.system
// Register connection service
IO(Http) ! Http.Bind(self, interface = "localhost", port = 9101)
def receive: Actor.Receive = {
case _: Http.Connected => sender ! Http.Register(self)
case HttpRequest(GET, Uri.Path("/timeout"), _, _, _) => {
Thread.sleep(3000)
sender ! HttpResponse(entity = HttpEntity("ok"))
}
case HttpRequest(GET, Uri.Path("/hello"), _, _, _) => {
sender ! HttpResponse(entity = HttpEntity("Helloworld"))
}
}
}
}
and My config for test:
spray {
can {
client {
response-chunk-aggregation-limit = 0
connecting-timeout = 1s
request-timeout = 1s
}
host-connector {
max-retries = 0
}
}
}
I found that in both cases, the "conn" object is the same.
So I guess when RequestTimeoutException happens, spray put back the conn to the pool (by default 4?) and the next case will use the same conn but at this time, this conn is keep alive, so the server will treat it as chunked request.
If I put some sleep in the second case, it will just passed.
So I guess I must close the conn when got RequestTimeoutException and make sure the second case use a fresh new connection, right?
How should I do? Any configurations?
Thanks
Leon
You should not block inside an Actor (your MockServer). When it is blocked, it is unable to respond to any messages. You can wrap the Thread.sleep and response inside a Future. Or even better: use the Akka Scheduler. Be sure to assign the sender to a val because it may change when you respond to the request asynchronously. This should do the trick:
val savedSender = sender()
context.system.scheduler.scheduleOnce(3 seconds){
savedSender ! HttpResponse(entity = HttpEntity("ok"))
}
I'm writing a client and server applications using Akka Tcp and I'm having a issue with a high throughput.
When I write too many messages on the client side, I'm having too many CommandFailed messages and I can't figure out why...
Here's my server:
class Server(listener: ActorRef) extends Actor {
import Tcp._
import context.system
IO(Tcp) ! Bind(self, new InetSocketAddress("localhost", 9090))
def receive = {
case CommandFailed(_: Bind) => {
println("command failed error")
context stop self
}
case c#Tcp.Connected(remote, local) =>
listener ! GatlingConnected(c.remoteAddress.toString)
println("Connected: " + c.remoteAddress)
val handler = context.actorOf(Props(classOf[ServerHandler], listener, c.remoteAddress.toString))
val connection = sender
connection ! Register(handler)
}
}
class ServerHandler(listener: ActorRef, remote: String) extends Actor {
import Tcp._
override def receive: Receive = {
case Received(data) => listener ! data.utf8String
case PeerClosed => {
listener ! Finished(remote)
context stop self
}
}
}
Message and Finished are just case classes that I've createad.
Here's the client (where I think its the source of this problem):
private class TCPMessageSender(listener: ActorRef) extends BaseActor {
final val MESSAGE_DELIMITER = "\n"
val buffer = new ListBuffer[Any]
val failedMessages = new ListBuffer[Write]
IO(Tcp) ! Connect(new InetSocketAddress(configuration.data.tcp.host, configuration.data.tcp.port))
override def receive = {
case msg # (_: UserMessage | _: GroupMessage | _: RequestMessage) =>
logger.warn(s"Received message ($msg) before connected. Buffering...")
buffer += msg
case CommandFailed(_: Connect) =>
logger.warn("Can't connect. All messages will be ignored")
listener ! Terminate
context stop self
case c # Connected(remote, local) =>
logger.info("Connected to " + c.remoteAddress)
val connection = sender
connection ! Register(self)
logger.info("Sending previous received messages: " + buffer.size)
buffer.foreach(msg => {
val msgString: String = JsonHelper.toJson(Map[String, Any]("message_type" -> msg.getClass.getSimpleName, "message" -> msg))
connection ! Write(ByteString(msgString + MESSAGE_DELIMITER))
})
buffer.clear
logger.info("Sent")
context become {
case msg # (_: UserMessage | _: GroupMessage | _: RequestMessage) =>
val msgString: String = JsonHelper.toJson(Map[String, Any]("message_type" -> msg.getClass.getSimpleName, "message" -> msg))
logger.trace(s"Sending message: $msgString")
connection ! Write(ByteString(msgString + MESSAGE_DELIMITER))
case CommandFailed(w: Write) =>
logger.error("Command failed. Buffering message...")
failedMessages += w
connection ! ResumeWriting
case CommandFailed(c) => logger.error(s"Command $c failed. I don't know what to do...")
case Received(data) =>
logger.warn(s"I am not supposed to receive this data: $data")
case "close" =>
connection ! Close
case _: ConnectionClosed =>
logger.info("Connection closed")
context stop self
case WritingResumed => {
logger.info("Sending failed messages")
failedMessages.foreach(write => connection ! write)
failedMessages.clear
}
}
}
}
Sometimes I receive a lot of CommandFailed messages and I call ResumeWrite and never receive a WritingResumed (and the connection never closes in this cases). Am I doing something wrong?
I think the problem is that when you register your actor sending the Register message, you also have to set the useResumeWriting parameter to true:
connection ! Register(handler, false, true)
The doc of resumeWriting command states:
When `useResumeWriting` is in effect as was indicated in the [[Tcp.Register]] message
then this command needs to be sent to the connection actor in order to re-enable
writing after a [[Tcp.CommandFailed]] event. All [[Tcp.WriteCommand]] processed by the
connection actor between the first [[Tcp.CommandFailed]] and subsequent reception of
this message will also be rejected with [[Tcp.CommandFailed]].