StreamTcpException while under server stress - scala

Question

I have a service using Akka HTTP that I have been doing some load testing on. Under stress, I've found that my service will ocassionally run into StreamTcpException when calling other service endpoints.
I create one flow for each endpoint which is shared by all of my actors. I am using something like this:
//this is done only once
val connectionFlow = Http(sys).outgoingConnection("host_name")
...
//each actor does this
val response = Source.single(HttpRequest(...)).via(connectionFlow).runWith(Sink.head)
I use Apache JMeter to load test my service, and with 40 threads, it typically takes 2000-4000 requests before I see my first error message. With 10 threads, it took me 9000 requests before I saw it.
The message looks like:
akka.stream.StreamTcpException: Tcp command [Connect(<host_here>/<ip_here>,None,List(),Some(10 seconds),true)] failed
I actually have 4 separate flows for 4 different endpoints my service relies on. I usually see StreamTcpException from all four if my service fails.
Anyone have any ideas why this is happening? Thanks in advance.

Answer 1

I face same issue previously, for me it is working as expected.
val httpClient: Flow[HttpRequest, HttpResponse, Future[Http.OutgoingConnection]] = Http().outgoingConnection(host,port,None,connSetts)
Problem place 👍 Source.single(request).via(httpClient)
It is unable to connect my host so I use proxy server for this.
val httpsProxyTransport : ClientTransport = ClientTransport.httpsProxy(InetSocketAddress.createUnresolved("proxyservicesHost", 90))
val connSetts: ClientConnectionSettings = ClientConnectionSettings(system).withTransport(httpsProxyTransport).withIdleTimeout(Duration(180, "second"))
val httpClient: Flow[HttpRequest, HttpResponse, Future[Http.OutgoingConnection]] = Http().outgoingConnection(host,port,None,connSetts)
Then I pass the httpclient in .via(httpclient), it works.