I am trying to set up a simple server/client akka (using Akka 2.0.3) application, but it failed to connect. Beforehand here is the basic code:
import com.typesafe.config.ConfigFactory
import akka.actor.Actor
import akka.actor.ActorSystem
import akka.actor.Props
class Server extends Actor {
def receive = {
case s: String => println("Got " + s)
}
}
val serverSystem = ActorSystem("server", ConfigFactory.load(ConfigFactory.parseString("""
akka {
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
remote {
transport = "akka.remote.netty.NettyRemoteTransport"
netty {
hostname = "localhost"
port = 5678
}
}
}
""")))
val server = serverSystem.actorOf(Props[Server], name = "server")
Thread.sleep(500)
println("started")
Thread.sleep(500)
val clientSystem = ActorSystem("client", ConfigFactory.load(ConfigFactory.parseString("""
akka {
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
}
""")))
val remoteServer = clientSystem.actorFor("akka://server#XXX:5678/user/server")
remoteServer ! "HEY"
Thread.sleep(3000)
clientSystem.shutdown
serverSystem.shutdown
I know that the configurations should be placed in external files.
If you replace XXX with localhost it works:
started
Got HEY
But if I used my external (resolved) IP (PC behind home router) for XXX the HEY message never arrives. I thought it is due to some firewall problem and forwarded the related TCP and UDP ports at my router and also opened/allowed them at my Windows firewall. So after that the following code worked (also XXX replaced with my external IP). A started ServerTest can be connected by a ClientTest:
import java.net.ServerSocket
object ServerTest extends App {
println("START server")
val ss = new ServerSocket(5678)
val s = ss.accept()
println(s)
println("END")
}
import java.net.Socket
object ClientTest extends App {
println("START client")
val s = new Socket("XXX", 5678)
println(s)
println("END")
}
So it´s not a port/firewall problem, isn´t it?! So where is the problem???
localhost usually means 127.0.0.1, which is only one of the possibly many interfaces (cards) in a computer. The server binding to localhost won't receive connections connecting to the other interfaces (including the one with the external address).
You should either specify the external address in the server, or 0.0.0.0 which means "bind to all interfaces".
Related
I am trying to setup a simple process with Spark Streaming, using Apache Bahir to connect to Akka. I tried to follow their example together with this older one. I have a simple forwarder actor
class ForwarderActor extends ActorReceiver {
def receive = {
case data: MyData => store(data)
}
}
and I create a stream with
val stream = AkkaUtils.createStream[RSVP](ssc, Props[ForwarderActor], actorName)
the configuration looks like this:
akka {
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "localhost"
port = 7777
}
}
}
and my problem is: how do I send messages to the Forwarder actor? Maybe I don't understand how Akka Remote is used in this case. When the app starts, I see a log
[akka.remote.Remoting] Remoting started; listening on addresses :[akka.tcp://test#localhost:7777]
and later on I see
[akka.remote.Remoting] Remoting now listens on addresses: [akka.tcp://streaming-actor-system-0#192.168.192.7:52369]
Which seems to remind to the description in the ScalaDoc:
/**
* A default ActorSystem creator. It will use a unique system name
* (streaming-actor-system-<spark-task-attempt-id>) to start an ActorSystem that supports remote
* communication.
*/
All in all I am not sure how I am supposed to send messages to the Forwarder actor. Thanks for any help!
Akka actors can send messages to other Akka actors running on a remote JVM. So... when the sender actor needs to know the address of the intended receiver actor.
AkkaUtil (Bahir) enables you to create a spark-stream from the messages that a ReceiverActor receives. But, where is is going to receive messages from ? Well... some remote actor. And to send messages this remote actor is going to need the address of your ReceiverActor which is running in your spark-application.
In general, you can not be too sure about the ip which will be running your spark application. So, we will make it so that the actor running with spark will tell the producer actor its reference and request it to send its things.
Just make sure that both applications are written using same version of Scala and are running the same JRE.
Now... lets first write the actor who will be the data source,
import akka.actor.{Actor, ActorRef, ActorLogging, ActorSystem, Props}
import akka.actor.Actor.Receive
import com.typesafe.config.{Config, ConfigFactory}
case class SendMeYourStringsRequest(requesterRef: ActorRef)
case class RequestedString(s: String)
class MyActor extends Actor with ActorLogging {
val theListOfMyStrings = List("one", "two", "three")
override def receive: Receive = {
case SendMeYourStringsRequest(requesterRef) => {
theListOfMyStrings.foreach(s => {
requesterRef ! RequestedString(s)
})
}
}
}
object MyApplication extends App {
val config = ConfigFactory.parseString(
"""
|akka{
| actor {
| provider = remote
| }
| remote {
| enabled-transports = ["akka.remote.netty.tcp"]
| untrusted-mode = off
| netty.tcp {
| hostname="my-ip-address"
| port=18000
| }
| }
|}
""".stripMargin
)
val actorSystem = ActorSystem("my-actor-system", config)
var myActor = actorSystem.actorOf(Props(classOf[MyActor]), "my-actor")
}
Now... lets write our simple spark app,
import akka.actor.{Actor, ActorRef, ActorLogging, ActorSystem, Props}
import akka.actor.Actor.Receive
import com.typesafe.config.{Config, ConfigFactory}
import org.apache.spark.SparkConf
import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.streaming.akka.{ActorReceiver, AkkaUtils}
case class SendMeYourStringsRequest(requesterRef: ActorRef)
case class RequestedString(s: String)
class YourStringRequesterActor extends ActorReceiver {
def receive = {
case RequestedString(s) => store(s)
}
override def preStart(): Unit = {
val myActorPath = ActorPath.fromString("akka.tcp://my-actor-system#my-ip-address:18000/user/my-actor")
val myActorSelection = context.actorSelection(myActorPath)
myActorSelection ! SendMeYourStringsRequest(self)
}
}
object YourSparkApp {
def main(args: Array[String]): Unit = {
val sparkConf = new SparkConf().setAppName("ActorWordCount")
if (!sparkConf.contains("spark.master")) {
sparkConf.setMaster("local[2]")
}
val ssc = new StreamingContext(sparkConf, Seconds(2))
val stringStream = AkkaUtils.createStream[String](
ssc,
Props(classOf[YourStringRequesterActor]),
"your-string-requester-actor"
)
stringStream.foreach(println)
}
}
Note :: Just take care of my-ip-address. If there are any other problems, please let me know in comments.
I try to redirect/forward a TCP stream to another Sink with Akka 2.4.3.
The program should open a server socket, listen for incoming connections and then consume the tcp stream. Our sender does not expect/accept replies from us so we never send back anything - we just consume the stream. After framing the tcp stream we need to transform the bytes into something more useful and send it to the Sink.
I tried the following so far but i struggle especially with the part how to not sending tcp packets back to the sender and to properly connect the Sink.
import scala.util.Failure
import scala.util.Success
import akka.actor.ActorSystem
import akka.event.Logging
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.Sink
import akka.stream.scaladsl.Tcp
import akka.stream.scaladsl.Framing
import akka.util.ByteString
import java.nio.ByteOrder
import akka.stream.scaladsl.Flow
object TcpConsumeOnlyStreamToSink {
implicit val system = ActorSystem("stream-system")
private val log = Logging(system, getClass.getName)
//The Sink
//In reality this is of course a real Sink doing some useful things :-)
//The Sink accept types of "SomethingMySinkUnderstand"
val mySink = Sink.ignore;
def main(args: Array[String]): Unit = {
//our sender is not interested in getting replies from us
//so we just want to consume the tcp stream and never send back anything to the sender
val (address, port) = ("127.0.0.1", 6000)
server(system, address, port)
}
def server(system: ActorSystem, address: String, port: Int): Unit = {
implicit val sys = system
import system.dispatcher
implicit val materializer = ActorMaterializer()
val handler = Sink.foreach[Tcp.IncomingConnection] { conn =>
println("Client connected from: " + conn.remoteAddress)
conn handleWith Flow[ByteString]
//this is neccessary since we use a self developed tcp wire protocol
.via(Framing.lengthField(4, 0, 65532, ByteOrder.BIG_ENDIAN))
//here we want to map the raw bytes into something our Sink understands
.map(msg => new SomethingMySinkUnderstand(msg.utf8String))
//here we like to connect our Sink to the Tcp Source
.to(mySink) //<------ NOT COMPILING
}
val tcpSource = Tcp().bind(address, port)
val binding = tcpSource.to(handler).run()
binding.onComplete {
case Success(b) =>
println("Server started, listening on: " + b.localAddress)
case Failure(e) =>
println(s"Server could not bind to $address:$port: ${e.getMessage}")
system.terminate()
}
}
class SomethingMySinkUnderstand(x:String) {
}
}
Update: Add this to your build.sbt file to get necessary deps
libraryDependencies += "com.typesafe.akka" % "akka-stream_2.11" % "2.4.3"
handleWith expects a Flow, i.e. a box with an unconnected inlet and an unconnected outlet. You effectively provide a Source, because you connected the Flow with a Sink by using the to operation.
I think you could do the following:
conn.handleWith(
Flow[ByteString]
.via(Framing.lengthField(4, 0, 65532, ByteOrder.BIG_ENDIAN))
.map(msg => new SomethingMySinkUnderstand(msg.utf8String))
.alsoTo(mySink)
.map(_ => ByteString.empty)
.filter(_ => false) // Prevents sending anything back
)
Alternative (and in my view cleaner) way to code it (AKKA 2.6.x), that will also emphasize of the fact, that you do not do any outbound flow, would be:
val receivingPipeline = Flow
.via(framing)
.via(decoder)
.to(mySink)
val sendingNothing = Source.never[ByteString]()
conn.handleWith(Flow.fromSinkAndSourceCoupled(receivingPiline, sendingNothing))
I am trying to run a server and remote actor. The two are up and running successfully. However, the server does not receive the message sent by the remote worker.
import akka.actor._
import akka.actor.Props
import akka.event.Logging
import com.typesafe.config.ConfigFactory
import java.security.MessageDigest
import akka.actor.{Actor, ActorSystem, Props}
import akka.routing.RoundRobinRouter
object Project {
// Define cases
case class register()
case class remoteWorkerActive()
// Main that accepts String argument to determine if the actor
// is the server or a worker
def main(args: Array[String]) {
println("I have " + args.length + " argument(s)")
println(args(0))
// Declare configurations for server and worker remote akka actors
//Attach configuration file of the Server
val serverConfiguration = ConfigFactory.parseString(
"""
akka{
actor{
provider = "akka.remote.RemoteActorRefProvider"
}
remote{
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp{
hostname = "127.0.0.1"
port = 2575
}
}
}""")
//Attach configuration file of the Worker
val workerConfiguration = ConfigFactory.parseString(
"""
akka{
actor{
provider = "akka.remote.RemoteActorRefProvider"
}
remote{
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp{
hostname = "127.0.0.1"
port = 0
}
}
}""")
// Based on the input argument, declare an actor system as either
// Server or Worker
if (!args(0).isEmpty) {
//Check if the argument is a valid IP address for a worker
if (args(0).contains('.')) {
//Create the Worker ActorSystem with the above configuration
val system = akka.actor.ActorSystem(
"Remote", ConfigFactory.load(workerConfiguration))
//Create the worker actor
val remote = system.actorOf(Props(
new Remote(args(0))), name = "remote")
remote ! register()
println("This actor is a worker")
}
else {
//Create the Server ActorSystem with the above configuration
val system = akka.actor.ActorSystem(
"Server", ConfigFactory.load(serverConfiguration))
//Create the server actor
val server = system.actorOf(Props[Server], name = "server")
println("This actor is a server")
server ! "test successful";
}
}
}
class Server extends Actor {
def receive = {
case remoteWorkerActive() =>
println("Registration of worker successful")
sender ! "You are a registered worker"
case msg: String =>
println(s"'$msg'")
case _ =>
println("Invalid message")
}
}
class Remote(ip_address: String) extends Actor {
println(ip_address)
val master = context.actorSelection(
"akka.tcp://Server#" + ip_address + ":2575/user/Server")
def receive = {
case register() =>
println("Trying to register with the server")
master ! remoteWorkerActive()
println("Message sent")
case msg: String =>
println(s"'$msg'")
}
}
}
What am I doing wrong? Also, is there any way I can track the messages being sent? Mostly for debugging.
You have:
val server = system.actorOf(Props[Server], name = "server")
and:
val master = context.actorSelection(
"akka.tcp://Server#" + ip_address + ":2575/user/Server")
If Akka ActorPaths are case sensitive, this will fail. Try name = "Server"
I'm trying to send messages to a remote actor, but failed.
My main code is:
RemoteActorDemo.scala
import akka.actor.{Actor, ActorSystem, Props}
object RemoteActorDemo extends App {
val system = ActorSystem("RemoteActorSystem")
val actor = system.actorOf(Props[RemoteActor], name = "RemoteActor")
actor ! "Remote Actor is alive"
}
class RemoteActor extends Actor {
override def receive: Receive = {
case msg =>
println("### RemoteActor received message: " + msg)
sender ! "Hello from RemoteActor"
}
}
With application.conf:
akka {
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "127.0.0.1"
port = 5150
}
}
}
And LocalActorDemo.scala:
import akka.actor.{Actor, ActorSystem, Props}
object LocalActorDemo extends App {
val system = ActorSystem("ActorDemo")
val localActor = system.actorOf(Props[LocalActor])
localActor ! "Start"
}
class LocalActor extends Actor {
val remote = context.actorSelection("akka.tcp://RemoteActorSystem#127.0.0.1:5150/user/RemoteActor")
override def receive: Receive = {
case "Start" =>
println("### LocalActor started")
remote ! "Hello from LocalActor"
case msg => println("*** LocalActor receives msg: " + msg)
}
}
The problem is the local actor can't connect the remote one. It prints in console:
### LocalActor started
[INFO] [10/05/2015 20:57:57.334] [ActorDemo-akka.actor.default-dispatcher-4] [akka://ActorDemo/deadLetters]
Message [java.lang.String] from Actor[akka://ActorDemo/user/$a#-11944341] to Actor[akka://ActorDemo/deadLetters]
was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration
settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
I'm new to akka, not sure where is wrong.
You can see the demo project here: https://github.com/freewind/remote-actors-demo, you can just clone and run it as "README" describes.
Add an application.conf in your local subproject with content like this:
akka {
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "127.0.0.1"
port = 0
}
}
}
As the official document says:
To enable remote capabilities in your Akka project you should, at a minimum, add the following changes to your application.conf file
This applied to the client side of a remoting system as well.
I've a bunch of existing REST services (#1 and #2 below) that are running on different endpoints that are used internally only. Now I want to expose some of these REST APIs (API-1 and API-2) externally using Spray because this external endpoint will also provide some additional APIs (API-3, API-4).
Is there a simple/recommended way to forward the external REST requests to my new endpoint to existing REST endpoints?
It sounds like what you want is the proposed proxyTo directive:
path("foo") {
get {
proxyTo("http://oldapi.example.com")
}
}
(or, more likely, proxyToUnmatchedPath). There is an issue open for it:
https://github.com/spray/spray/issues/145
Looks like somebody has been working on this; here is a commit in a Spray fork:
https://github.com/bthuillier/spray/commit/d31fc1b5e1415e1b908fe7d1f01f364a727e2593
But the commit appears not yet to be in the master Spray repo. You could ask about its status on the issue page.
Also, here is a blog post from CakeSolutions about how you can do the proxying manually:
http://www.cakesolutions.net/teamblogs/http-proxy-with-spray
A comment on that page points out that Spray has an undocumented thing called ProxySettings, and points to the following tests for it:
https://github.com/spray/spray/blob/master/spray-can-tests/src/test/scala/spray/can/client/ProxySpec.scala
UPDATE; Soumya has asked the Spray team about this on the spray-user Google Group:
https://groups.google.com/forum/#!topic/spray-user/MlUn-y4X8RE
I was able to proxy a single service with the help of the CakeSolution blog. In the following example, the proxy is running on http://localhost:20000 and the actual REST endpoint is running at http://localhost:7001.
Not sure how proxy multiple services using this approach.
I like #cmbaxter's solution of using Nginx as the proxy but I'm still curious if there is a way to extend the following approach to do it in Spray.
import akka.actor.{ActorRef, Props}
import akka.io.IO
import akka.util.Timeout
import spray.can.Http
import spray.can.Http.ClientConnectionType
import spray.http.HttpResponse
import spray.routing.{RequestContext, HttpServiceActor, Route}
import scala.concurrent.duration._
import akka.pattern.ask
object ProxyRESTService {
def main(args: Array[String]) {
//create an actor system
implicit val actorSystem = akka.actor.ActorSystem("proxy-actor-system")
implicit val timeout: Timeout = Timeout(5 seconds)
implicit val dis = actorSystem.dispatcher
//host on which proxy is running
val proxyHost = "localhost"
//port on which proxy is listening
val proxyPort = 20000
//host where REST service is running
val restServiceHost = "localhost"
//port where REST service is running
val restServicePort = 7001
val setup = Http.HostConnectorSetup(
proxyHost,
proxyPort,
connectionType = ClientConnectionType.Proxied(restServiceHost, restServicePort)
)
IO(Http)(actorSystem).ask(setup).map {
case Http.HostConnectorInfo(connector, _) =>
val service = actorSystem.actorOf(Props(new ProxyService(connector)))
IO(Http) ! Http.Bind(service, proxyHost, port = proxyPort)
}
}
}
.
class ProxyService(connector: ActorRef) extends HttpServiceActor {
implicit val timeout: Timeout = Timeout(5 seconds)
implicit def executionContext = actorRefFactory.dispatcher
val route: Route = (ctx: RequestContext) => ctx.complete(connector.ask(ctx.request).mapTo[HttpResponse])
def receive: Receive = runRoute(route)
}