So I want to write some network code that appears to be blocking, without actually blocking a thread. I'm going to send some data out on the wire, and have a 'queue' of responses that will come back over the network. I wrote up a very simple proof of concept, inspired by the producer/consumer example on the actor tutorial found here: http://www.scala-lang.org/node/242
The thing is, using receive appears to take up a thread, and so I'm wondering if theres anyway to not take up a thread and still get the 'blocking feel'. Heres my code sample:
import scala.actors.Actor._;
import scala.actors.Actor;
case class Request(val s:String);
case class Message(val s:String);
class Connection {
private val act:Actor = actor {
loop {
react {
case m:Message => receive { case r:Request => reply { m } }
}
}
}
def getNextResponse(): Message = {
return (act !? new Request("get")).asInstanceOf[Message];
}
//this would call the network layer and send something over the wire
def doSomething() {
generateResponse();
}
//this is simulating the network layer getting some data back
//and sending it to the appropriate Connection object
private def generateResponse() {
act ! new Message("someData");
act ! new Message("moreData");
act ! new Message("even more data");
}
}
object runner extends Application {
val conn = new Connection();
conn.doSomething();
println( conn.getNextResponse());
println(conn.getNextResponse());
println(conn.getNextResponse());
}
Is there a way to do this without using the receive, and thereby making it threadless?
You could directly rely on react which should block and release the thread:
class Connection {
private val act:Actor = actor {
loop {
react {
case r:Request => reply { r }
}
}
}
[...]
I expect that you can use react rather than receive and not have actors take up threads like receive does. There is thread on this issue at receive vs react.
Related
I am building an application using Tumblr's new Colossus framework (http://tumblr.github.io/colossus/). There is still limited documentation on it (and the fact that I'm still very new to Akka doesn't help), so I was wondering if someone could chime in on whether my approach is correct.
The application is simple and consists of two key components:
A thin web service layer that will queue tasks into Redis
A background worker which will poll the same Redis instance for available tasks and process them as they become available
I made a simple example to demonstrate that my concurrency model will work (and it does), which I posted below. However, I would like to make sure that there is not a more idiomatic way to do this.
import colossus.IOSystem
import colossus.protocols.http.Http
import colossus.protocols.http.HttpMethod.Get
import colossus.protocols.http.UrlParsing._
import colossus.service.{Callback, Service}
import colossus.task.Task
object QueueProcessor {
implicit val io = IOSystem() // Create separate IOSystem for worker
Task { ctx =>
while(true) {
// Below code is for testing purposes only. This is where the Redis loop will live, and will use a blocking call to get the next available task
Thread.sleep(5000)
println("task iteration")
}
}
def ping = println("starting") // Method to launch this processor
}
object Main extends App {
implicit val io = IOSystem() // Primary IOSystem for the web service
QueueProcessor.ping // Launch worker
Service.serve[Http]("app", 8080) { ctx =>
ctx.handle { conn =>
conn.become {
case req#Get on Root => Callback.successful(req.ok("Here"))
// The methods to add tasks to the queue will live here
}
}
}
}
I tested the above model and it works. The background loop continues running while the service happily accepts requests. But, I think that there might be a better way to do this with workers (nothing found in documentation), or perhaps Akka Streams?
I got it working with something that seems semi-idiomatic to me. However, new answers & feedback are still welcomed!
class Processor extends Actor {
import scala.concurrent.ExecutionContext.Implicits.global
override def receive = {
case "start" => self ! "next"
case "next" => {
Future {
blocking {
// Blocking call here to wait on Redis (BRPOP/BLPOP)
self ! "next"
}
}
}
}
}
object Main extends App {
implicit val io = IOSystem()
val processor = io.actorSystem.actorOf(Props[Processor])
processor ! "start"
Service.serve[Http]("app", 8080) { ctx =>
ctx.handle { conn =>
conn.become {
// Queue here
case req#Get on Root => Callback.successful(req.ok("Here\n"))
}
}
}
}
Here is my simple routing application:
object Main extends App with SimpleRoutingApp {
implicit val system = ActorSystem("my-system")
startServer(interface = "0.0.0.0", port = System.getenv("PORT").toInt) {
import format.UsageJsonFormat._
import spray.httpx.SprayJsonSupport._
path("") {
get {
complete("OK")
}
} ~
path("meter" / JavaUUID) {
meterUUID => pathEnd {
post {
entity(as[Usage]) {
usage =>
// execute some logic asynchronously
// do not wait for the result
complete("OK")
}
}
}
}
}
}
What I want to achieve is to execute some logic asynchronously in my path directive, do not wait for the result and return immediately HTTP 200 OK.
I am quite new to Scala and spray and wondering if there is any spray way to solve this specific problem. Otherwise I would go into direction of creating Actor for every request and letting it to do the job. Please advice.
There's no special way of handling this in spray: simply fire your async action (a method returning a Future, a message sent to an actor, whatever) and call complete right after.
def doStuffAsync = Future {
// literally anything
}
path("meter" / JavaUUID) { meterUUID =>
pathEnd {
post {
entity(as[Usage]) { usage =>
doStuffAsync()
complete("OK")
}
}
}
}
Conversely, if you need to wait for an async action to complete before sending the response, you can use spray-specific directives for working with Futures or Actors.
I have started working with Netty and have a question. I am bootstrapping Netty as shown in the examples.
def connect {
try {
bootstrap.group(group)
.channel(classOf[NioSocketChannel])
.remoteAddress(new InetSocketAddress(host, port))
.handler(new ChannelInitializer[SocketChannel] {
override def initChannel(ch: SocketChannel) {
ch.pipeline.addLast(new StringDecoder(CharsetUtil.UTF_8))
.addLast(new NettyConnectionChannelInbound)
}
})
val future: ChannelFuture = bootstrap.connect.sync
future.channel.closeFuture.sync
} finally {
group.shutdownGracefully
}
}
I have another class with the channelRead method in it
class NettyConnectionChannelInbound extends ChannelInboundHandlerAdapter {
override def channelRead(ctx: ChannelHandlerContext, msg: Any) {
println(msg)
}
}
When I run my application
val nc = new NettyConnection
nc.connect
println("After connection")
It doesn't ever seem to get to the println after the connect call. Is this expected or am I doing something wrong? Does it just sit and wait for incoming messages to come in if I don't close the socket after receiving something?
That is because closeFuture().sync() will block until the channel is closed. So I don't think this is what you want...
looks like you are using netty python. But it seems somewhat strange to me that your bootstrap.connect does not specify host & port...(could that be due to pseudo-code grabbed from online)
Of course it needs to be blocking after connection otherwise your program would quit with connection being forced to close.
Your channelRead will not be invoked until a message is sent from the other end of the network.
I have an Actor that is similar to the following Actor in function.
case class SupervisingActor() extends Actor {
protected val processRouter = //round robin router to remote workers
override def receive = {
case StartProcessing => { //sent from main or someplace else
for (some specified number of process actions ){
processRouter ! WorkInstructions
}
}
case ProcessResults(resultDetails) => { //sent from the remote workers when they complete their work
//do something with the results
if(all of the results have been received){
//*********************
self ! EndProcess //This is the line in question
//*********************
}
}
case EndProcess {
//do some reporting
//shutdown the ActorSystem
}
}
}
}
How can I verify the EndProcess message is sent to self in tests?
I'm using scalatest 2.0.M4, Akka 2.0.3 and Scala 1.9.2.
An actor sending to itself is very much an intimiate detail of how that actor performs a certain function, hence I would rather test the effect of that message than whether or not that message has been delivered. I’d argue that sending to self is the same as having a private helper method on an object in classical OOP: you also do not test whether that one is invoked, you test whether the right thing happened in the end.
As a side note: you could implement your own message queue type (see https://doc.akka.io/docs/akka/snapshot/mailboxes.html#creating-your-own-mailbox-type) and have that allow the inspection or tracing of message sends. The beauty of this approach is that it can be inserted purely by configuration into the actor under test.
In the past, I have overridden the implementation for ! so that I could add debug/logging. Just call super.! when you're done, and be extra careful not to do anything that would throw an exception.
I had the same issue with an FSM actor. I tried setting up a custom mailbox as per the accepted answer but a few minutes didn't get it working. I also attempted to override the tell operator as per another answer but that was not possible as self is a final val. Eventually I just replaced:
self ! whatever
with:
sendToSelf(whatever)
and added that method into the actor as:
// test can override this
protected def sendToSelf(msg: Any) {
self ! msg
}
then in the test overrode the method to capture the self sent message and sent it back into the fsm to complete the work:
#transient var sent: Seq[Any] = Seq.empty
val fsm = TestFSMRef(new MyActor(x,yz) {
override def sendToSelf(msg: Any) {
sent = sent :+ msg
}
})
// yes this is clunky but it works
var wait = 100
while( sent.isEmpty && wait > 0 ){
Thread.sleep(10)
wait = wait - 10
}
fsm ! sent.head
first of all, i'm learning scala and new to the java world.
I want to create a console and run this console as a service that you could start and stop.
I was able to run a ConsoleReader into an Actor but i don't know how to stop properly the ConsoleReader.
Here is the code :
import eu.badmood.util.trace
import scala.actors.Actor._
import tools.jline.console.ConsoleReader
object Main {
def main(args:Array[String]){
//start the console
Console.start(message => {
//handle console inputs
message match {
case "exit" => Console.stop()
case _ => trace(message)
}
})
//try to stop the console after a time delay
Thread.sleep(2000)
Console.stop()
}
}
object Console {
private val consoleReader = new ConsoleReader()
private var running = false
def start(handler:(String)=>Unit){
running = true
actor{
while (running){
handler(consoleReader.readLine("\33[32m> \33[0m"))
}
}
}
def stop(){
//how to cancel an active call to ConsoleReader.readLine ?
running = false
}
}
I'm also looking for any advice concerning this code !
The underlying call to read a characters from the input is blocking. On non-Windows platform, it will use System.in.read() and on Windows it will use org.fusesource.jansi.internal.WindowsSupport.readByte.
So your challenge is to cause that blocking call to return when you want to stop your console service. See http://www.javaspecialists.eu/archive/Issue153.html and Is it possible to read from a InputStream with a timeout? for some ideas... Once you figure that out, have read return -1 when your console service stops, so that ConsoleReader thinks it's done. You'll need ConsoleReader to use your version of that call:
If you are on Windows, you'll probably need to override tools.jline.AnsiWindowsTerminal and use the ConsoleReader constructor that takes a Terminal (otherwise AnsiWindowsTerminal will just use WindowsSupport.readByte` directly)
On unix, there is one ConsoleReader constructor that takes an InputStream, you could provide your own wrapper around System.in
A few more thoughts:
There is a scala.Console object already, so for less confusion name yours differently.
System.in is a unique resource, so you probably need to ensure that only one caller uses Console.readLine at a time. Right now start will directly call readLine and multiple callers can call start. Probably the console service can readLine and maintain a list of handlers.
Assuming that ConsoleReader.readLine responds to thread interruption, you could rewrite Console to use a Thread which you could then interrupt to stop it.
object Console {
private val consoleReader = new ConsoleReader()
private var thread : Thread = _
def start(handler:(String)=>Unit) : Thread = {
thread = new Thread(new Runnable {
override def run() {
try {
while (true) {
handler(consoleReader.readLine("\33[32m> \33[0m"))
}
} catch {
case ie: InterruptedException =>
}
}
})
thread.start()
thread
}
def stop() {
thread.interrupt()
}
}
You may overwrite your ConsoleReader InputStream. IMHO this is reasonable well because of STDIN is a "slow" stream. Please improve example for your needs. This is only sketch, but it works:
def createReader() =
terminal.synchronized {
val reader = new ConsoleReader
terminal.enableEcho()
reader.setBellEnabled(false)
reader.setInput(new InputStreamWrapper(reader.getInput())) // turn on InterruptedException for InputStream.read
reader
}
with InputStream wrapper:
class InputStreamWrapper(is: InputStream, val timeout: Long = 50) extends FilterInputStream(is) {
#tailrec
final override def read(): Int = {
if (is.available() != 0)
is.read()
else {
Thread.sleep(timeout)
read()
}
}
}
P.S. I tried to use NIO - a lot of troubles with System.in (especially crossplatform). I returned to this variant. CPU load is near 0%. This is suitable for such interactive application.