Obtaining the client IP in Akka-http - scala

I am trying to write an Akka HTTP microservice (akka version 2.4.11, Scala version 2.11.8, both latest versions at time of writing) which is aware of the client service's IP (i.e., remote address), and I cannot get this to work.
I can create and run a service which says 'Hello!' using a route like this:
val routeHello: Route = path("SayHello") {
get {
entity(as[String]) {
body => complete {
HttpResponse(entity = HttpEntity("Hello!"))
}
}
}
}
I have constructed a similar route to the one above, which is extended so that it is aware of the client's IP address.
I noted that I need to edit the application.conf file and set 'remote-address-header = on' to enable the addition of a Remote-Address header holding the clients (remote) IP address. I have done this in case it is required.
Here is the route:
val routeHelloIp: Route = path("SayHelloIp") {
get {
// extractClientIp appears to be working as a filter
// instead of an extractor - why?
extractClientIp {
clientIp => {
entity(as[String]) {
body => complete {
HttpResponse(entity = HttpEntity("Hello!"))
}
}
}
}
}
}
However when I run this route, I get a message 'The requested resource could not be found.'.
It looks like I have got the Akka-http DSL syntactic sugar wrong in the example above. I would be grateful if you could put me on the right path!
EDIT:
I have tried the following program in response to Ramon's helpful answer. Unfortunately it does not compile and I cannot see what I need to do to make it compile.
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.Http.IncomingConnection
import java.net.InetSocketAddress
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.Sink
import akka.http.scaladsl.server.Directives._
import java.net.InetSocketAddress
object TestHttp {
def main(args: Array[String]) {
implicit val system = ActorSystem("my-system")
implicit val materializer = ActorMaterializer()
// allow connections from any IP
val interface = "0.0.0.0"
//from the question
def createRoute(address: InetSocketAddress) = path("SayHelloIp") {
get {
extractRequestEntity { entity =>
entity(as[String]) { body =>
complete(entity = s"Hello ${address.getAddress().getHostAddress()}")
}
}
}
}
Http().bind(interface).runWith(Sink foreach { conn =>
val address = conn.remoteAddress
conn.handleWithAsyncHandler(createRoute(address))
})
}
}
I have the following build.sbt to ensure that the latest version of Scala and akka-http are used:
import sbt.Keys._
name := "Find my IP"
version := "1.0"
scalaVersion := "2.11.8"
resolvers ++= Seq(
"Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
)
libraryDependencies ++= {
Seq(
"com.typesafe.akka" %% "akka-actor" % "2.4.11",
"com.typesafe.akka" %% "akka-stream" % "2.4.11",
"com.typesafe.akka" %% "akka-http-experimental" % "2.4.11",
"com.typesafe.akka" %% "akka-http-core" % "2.4.11"
)
}
I get the following compile-time errors:
[error] /Users/tilopa/temp/akka-test/src/main/scala/Test.scala:24: akka.http.scaladsl.model.RequestEntity does not take parameters
[error] entity(as[String]) { body =>
[error] ^
[error] /Users/tilopa/temp/akka-test/src/main/scala/Test.scala:25: reassignment to val
[error] complete(entity = s"Hello ${address.getAddress().getHostAddress()}")
[error] ^
[error] two errors found
[error] (compile:compileIncremental) Compilation failed

Using extractClientIp
extractClientIp is not working for you because the sender has not specified one of the required header fields. From the documentation:
Provides the value of X-Forwarded-For, Remote-Address, or X-Real-IP
headers as an instance of RemoteAddress.
You just have to turn on the right setting in your sender:
The akka-http server engine adds the Remote-Address header to every
request automatically if the respective setting
akka.http.server.remote-address-header is set to on. Per default it is
set to off.
generic solution
If you want this to work for any HttpRequest, not just the ones with the correct header settings, then you have to use the bind method on an HttpExt instead of bindAndHandle:
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.Http.IncomingConnection
import java.net.InetSocketAddress
implicit val actorSystem : ActorSystem = ???
implicit val actorMat = ActorMaterializer()
//alow connections from any IP
val interface = "0.0.0.0"
//from the question
def createRoute(address : InetSocketAddress) = path("SayHelloIp") {
get {
extractRequestEntity { entity =>
entity(as[String]) { body =>
complete(entity = s"Hello ${address.getAddress().getHostAddress()}")
}
}
}
}
Http().bind(interface).runWith(Sink foreach { conn =>
val address = conn.remoteAddress
conn.handleWithAsyncHandler(createRoute(address))
})
Edit
As noted in the comments: since akka 10.0.13 use conn.handleWith.

HttpExt.bind() has been depreciated. Here's a workaround with connectionSource():
val https: HttpsConnectionContext = /*...*/
Http(actorSystem).
newServerAt(interface,httpsPort).
enableHttps(https).
connectionSource().
to(Sink.foreach { connection =>
println("New Connection from remote address :: ${connection.remoteAddress}")
// handle connection here
}).run().onComplete {
case Success(binding)=>
binding.addToCoordinatedShutdown(hardTerminationDeadline = 10.seconds)
log.info("{} HTTPS Server running {}",errorContext,binding)
case Failure(ex)=>
log.error("{} HTTPS Server failed {}",errorContext,ex.toString)
sys.exit(10)
}

Related

how to start server/client grpc using scalapb on spark?

i have a problem about running server/client using ScalaPB on spark.
its totally work fine while I running my code using "sbt run". i want running this code using spark coz next ill import my spark model to predict some label. but while I submit my jar to spark, they give me error like this.
Exception in thread "main" io.grpc.ManagedChannelProvider$ProviderNotFoundException:
No functional server found. Try adding a dependency on the grpc-netty artifact
this is my build.sbt
scalaVersion := "2.11.7"
PB.targets in Compile := Seq(
scalapb.gen() -> (sourceManaged in Compile).value
)
val scalapbVersion =
scalapb.compiler.Version.scalapbVersion
val grpcJavaVersion =
scalapb.compiler.Version.grpcJavaVersion
libraryDependencies ++= Seq(
// protobuf
"com.thesamet.scalapb" %% "scalapb-runtime" % scalapbVersion % "protobuf",
//for grpc
"io.grpc" % "grpc-netty" % grpcJavaVersion ,
"com.thesamet.scalapb" %% "scalapb-runtime-grpc" % scalapbVersion
)
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
using shade still not work
assemblyShadeRules in assembly := Seq(ShadeRule.rename("com.google.**" -> "shadegoogle.#1").inAll)
and this my main
import java.util.logging.Logger
import io.grpc.{Server, ServerBuilder}
import org.apache.spark.ml.tuning.CrossValidatorModel
import org.apache.spark.sql.SparkSession
import testproto.test.{Email, EmailLabel, RouteGuideGrpc}
import scala.concurrent.{ExecutionContext, Future}
object HelloWorldServer {
private val logger = Logger.getLogger(classOf[HelloWorldServer].getName)
def main(args: Array[String]): Unit = {
val server = new HelloWorldServer(ExecutionContext.global)
server.start()
server.blockUntilShutdown()
}
private val port = 50051
}
class HelloWorldServer(executionContext: ExecutionContext) {
self =>
private[this] var server: Server = null
private def start(): Unit = {
server = ServerBuilder.forPort(HelloWorldServer.port).addService(RouteGuideGrpc.bindService(new RouteGuideImpl, executionContext)).build.start
HelloWorldServer.logger.info("Server started, listening on " + HelloWorldServer.port)
sys.addShutdownHook {
System.err.println("*** shutting down gRPC server since JVM is shutting down")
self.stop()
System.err.println("*** server shut down")
}
}
private def stop(): Unit = {
if (server != null) {
server.shutdown()
}
}
private def blockUntilShutdown(): Unit = {
if (server != null) {
server.awaitTermination()
}
}
private class RouteGuideImpl extends RouteGuideGrpc.RouteGuide {
override def getLabel(request: Email): Future[EmailLabel] = {
val replay = EmailLabel(emailId = request.emailId, label = "aaaaa")
Future.successful(replay)
}
}
}
thanks
It looks like grpc-netty is not found when an uber jar is made. Instead of using ServerBuilder, change your code to use io.grpc.netty.NettyServerBuilder.

Exception while using Play WS to access a REST WebService

I want to use the Play WS Standalone (outside a Play Project) to call a simple Hello World RESTfull Web Service.
I'm Using an SBT Poject in Intelij IDEA whith this configuration :
name := "ScalaSbtProject"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies += "com.typesafe.play" % "play-ws_2.11" % "2.5.10"
Here's my code :
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import play.api.libs.ws.ahc.AhcWSClient
import scala.util.{Failure, Success}
object Test {
def main(args: Array[String]): Unit = {
import scala.concurrent.ExecutionContext.Implicits._
implicit val actor = ActorSystem()
implicit val materializer = ActorMaterializer()
val wsClient = AhcWSClient()
val response = wsClient
.url("http://localhost:8080/WebService_war_exploded/service/test")
.get()
response.onComplete {
case Success(re) => { println(re.body)}
case Failure(e) => { e.getStackTrace.foreach(println)}
}
wsClient.close()
}
}
which throws this Exception :
Response Is Failed :
http://localhost:8080
org.asynchttpclient.netty.channel.NettyConnectListener.onFailure(NettyConnectListener.java:160)
org.asynchttpclient.netty.request.NettyChannelConnector$1.onFailure(NettyChannelConnector.java:103)
org.asynchttpclient.netty.SimpleChannelFutureListener.operationComplete(SimpleChannelFutureListener.java:28)
org.asynchttpclient.netty.SimpleChannelFutureListener.operationComplete(SimpleChannelFutureListener.java:20)
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:507)
io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:486)
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:129)
io.netty.channel.nio.AbstractNioChannel.doClose(AbstractNioChannel.java:458)
io.netty.channel.socket.nio.NioSocketChannel.doClose(NioSocketChannel.java:235)
io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:632)
io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:611)
io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:554)
io.netty.channel.nio.NioEventLoop.closeAll(NioEventLoop.java:637)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:406)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
java.lang.Thread.run(Thread.java:745)

Get content from Akka ResponseEntity in Scala

I do a GET HTTP call to a rest service which returns a json. I would like to parse the json to a scala object but here I got stuck. I am using the Akka api and I can't manage to retrieve the content from the Akka's ResponseEntity
Here is my sbt file:
name := "ScalaHttp"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies ++={
val akkaV = "2.4.5"
Seq(
"com.typesafe.akka" %% "akka-http-core" % akkaV,
"com.typesafe.play" %% "play-json" % "2.4.0-M3"
)
}
And here is the app
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.HttpRequest
import akka.stream.ActorMaterializer
import scala.concurrent.ExecutionContext.Implicits.global
object Sender {
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
Http().singleRequest(HttpRequest(uri = "http://declinators.com/declinator?noun=komunikacja")) foreach {
y => println(y.entity)
println(y.entity.getContentType())
println(y.entity.contentType)
}
}
}
This prints:
HttpEntity.Strict(application/json,{"nominative":"komunikacja","genitive":"komunikacji","dative":"komunikacji","accusative":"komunikacjÄ™","instrumental":"komunikacjÄ…","locative":"komunikacji","vocative":"komunikacjo"})
application/json
application/json
Here come the questions:
1. Why ResponseEntity supplies getContentType() and contentType()? They return the same thing.
2. Getting the contentyType is easy, you have two ways for doing it, but how can I get the content itself, so I can play (i.e. parse it using play) with the json!
You can use entity.data.toString for Binary content type, or the following code piece
data.decodeString(nb.charset.value)
Please follow the HttpEntity.Strict.toString implementation here for detail:
https://github.com/akka/akka/blob/master/akka-http-core/src/main/scala/akka/http/scaladsl/model/HttpEntity.scala#L316

Load Balancing akka-http

I am using akka-http, my build.sbt configuration is:
scalaVersion := "2.11.7"
libraryDependencies += "com.typesafe.akka" % "akka-actor_2.11" % "2.4.2"
libraryDependencies += "com.typesafe.akka" % "akka-http-experimental_2.11" % "2.4.2"
libraryDependencies += "com.typesafe.akka" % "akka-http-spray-json-experimental_2.11" % "2.4.2"
libraryDependencies += "com.typesafe.akka" % "akka-slf4j_2.11" % "2.4.2"
I am exposing a simple REST api with only one GET url
foo is a function that returns a Future
implicit val actorSystem = ActorSystem("system", config)
implicit val actorMaterializer = ActorMaterializer()
val route: Route = {
get {
path("foo") {
complete { foo }
}
}
}
The web-service is expected to have a lot of calls, and I want to make the service redundant in case of failure, so I want to have two instances running simultaneously to handle all the requests.
1) what is the best way to have two intances of the web-service handling the requests simultaneously?, with an external load balancer or with some magic (wich I don't know) inside akka/akka-http ?
2) What are the principal parameters I have to tune up to improve perfomance?
The answer to this question demonstrates how to make an Actor call from within a Route.
If you combine that technique with the clustering capabilities within Akka you should be able to get the job done.
Have your Route send a message to a Router that will dispatch the message to 1 of N remotely deployed Actors (from your question it sounds like a round robin router is what you want).
class HttpResponseActor extends Actor {
def foo : HttpResponse = ??? // Not specified in question
override def receive = {
case _ : HttpRequest => foo
}
}
import akka.actor.{ Address, AddressFromURIString }
import akka.remote.routing.RemoteRouterConfig
val addresses = Seq(Address("akka.tcp", "remotesys", "otherhost", 1234),
AddressFromURIString("akka.tcp://othersys#anotherhost:1234"))
val routerRemote =
system.actorOf(RemoteRouterConfig(RoundRobinPool(5), addresses).props(Props[HttpResponseActor]))
The remote Actor responds with the HttpResponse. This Response can go through the Router or directly back to the Route.
The Route sticks the answer in a complete Directive to return back to the client.
val route =
get {
path("foo") {
onComplete((routerRemote ? request).mapTo[HttpResponse]) {
case Success(response) => complete(response)
case Failure(ex) => complete((InternalServerError, s"Actor not playing nice: ${ex.getMessage}"))
}
}
}

Basic Spray-Testkit usage to test a route does not work

I am trying to use spray route and want to test it with Spray-TestKit.
I am using :
- Scala 2.10.3
- Akka 2.3.3
- Spray 1.3.1
I create a trait extending HttpService, where I define a route :
trait MyService extends HttpService with CoreAccess {
import greentee.am.endpoint.tsmsp.tsmSPJsonSupport._
val myRoute = {
path("resources"/"ping") {
get {
complete(OK, "pong")
}
}
}
}
I deleted part of the route which was not relevant.
CoreAccess is a trait extending Actor, because I have methods in that trait access the ActorSystem. (I don't know who to retrieve ActorSelection from a trait without it extending an actor)
Then I create a test Specification
import MyService
import org.specs2.mutable.Specification
import spray.testkit.Specs2RouteTest
import spray.http.StatusCodes._
class RegistrationRouteSpecification extends Specification with Specs2RouteTest with MyService {
def actorRefFactory = system
"The EndPoint " should {
"return Pong to a Get request to the ping" in {
Get("/resources/ping") ~> myRoute ~> check {
status === OK
responseAs[String] === "pong"
}
}
}
}
When I try to execute the test, I get the following compilation error:
[info] Compiling 1 Scala source to /Users/IdeaProjects/endpoint/target/scala-2.10/test-classes...
[error] /Users/IdeaProjects/endpoint/src/test/scala/RegistrationRouteSpecification.scala:19: could not find implicit value for parameter ta: RegistrationRouteSpecification.this.TildeArrow[spray.routing.RequestContext,Unit]
[error] Get("/resources/ping") ~> myRoute ~> check {
[error] ^
[error] one error found
I answer my own question.
I corrected my Build.scala to use the following lines:
val scalaCheck = "org.scalacheck" %% "scalacheck" % Versions.scalaCheckVersion % "test"
val scalaTest = "org.scalatest" %% "scalatest" % "2.2.0" % "test"
Instead of using a simple '%' and supplying a dedicated version.