Scala dispatch reboot timeout - scala

I am using the dispatch reboot library version 0.9.5 (http://dispatch.databinder.net/Dispatch.html) in my project. Via sbt I have the following line:
libraryDependencies += "net.databinder.dispatch" %% "dispatch-core" % "0.9.5"
In the scala (2.9.2) repl (started using sbt console to get the appropriate dependency) and independent of my code I run the following session:
import dispatch._
import java.util.concurrent.TimeUnit._
val spoo = Http.threads(1).waiting( Duration(10, SECONDS ) )
(I believe that the third line sets up my own threadpool with one thread and a timeout of 10 seconds).
I then run this code repeatedly (in paste mode), to submit a future to fetch a particular url and then print the status code (asynchronously):
spoo(url("http://www.evapcool.com/products/commercial/")).either
.map {
case Right(r) => println( "S: " + r.getStatusCode())
case Left(e) => println( "E: " + e.toString ) }
Each time I run this line I wait for the status code to get printed before running the line again. For the first twenty to forty calls it works as expected. Then it reliably fails to report either a successful page reply or an exception. My assumption was that if this is caused by a timeout, I should expect the callback to fire after 10 seconds with the Left clause of the Either containing some form of timeout exception. But my experience is that this never comes.
Can anyone help tell me what I'm doing wrong?
Update
By the way, I am aware that there is a similar question (with answer) here but I am looking for the official (i.e. intended by the library author) way to handle timeouts - and it appears to me that this is what the waiting method is designed for

So an answer, although one I'm not enormously satisfied with - because it involves ignoring the rather nice looking waiting method on Http and playing directly with the async-http-client apis is this (inspired by the SO post linked in the question update):
val spoo = Http.threads( 1 ).configure
{ builder =>
builder.setRequestTimeoutInMs( 10000 )
builder.setConnectionTimeoutInMs( 10000 )
builder
}
My code now runs as expected. Ho hum...

Related

Gatling - Long Polling in a loop - asynchronous response - loop doesn't work as intended

This is my first question ever on this website, please bear with me patiently.
I'm trying to make an http long polling program for a project using Gatling. While crawling through many questions on stackoverflow, though I've been able to reunite separate concepts into a piece of syntactically correct code, sadly, it doesn't do what is intended to do.
When a status code of 200 is obtained after any request, the loop should break and the test would be considered as approved. If the status code is different to 200, it should keep the connection alive and polling, not failing the test.
When the .tryMax value is reached and all responses gave a status different to 200, the loop should break and the test should be considered as failed.
Using the difference operator (!=) doesn't work either, so then I took the decision to alternatively use .equals() and test the loop, to no avail.
Being new to both Gatling and Scala, I'm still trying to figure out what's wrong with this code, execution-wise:
def HttpPollingAsync() = {
asLongAs(session => session("statuss").validate[String].equals("200")) {
exec(
polling
.every(10 seconds)
.exec(
http("polling-async-response")
.post("/" + BaseURL + "/resource-async-response")
.headers(headers)
.body(RawFileBody("requestdata.json"))
.check(
status.is(200),
jsonPath("$.status").is("200"),
jsonPath("$.status").saveAs("statuss")
))
).exec(polling.stop)
}
}
val scn = scenario("asyncpolling")
.tryMax(60){
exec(HttpPollingAsync())
}
setUp(scn.inject(atOnceUsers(10))).protocols(httpProtocol)
The exception I get when running this piece of code is (it's just syntactically correct):
Exception in thread "main" java.lang.UnsupportedOperationException: There were no requests sent during the simulation, reports won't be generated
at io.gatling.charts.report.ReportsGenerator.generateFor(ReportsGenerator.scala:49)
at io.gatling.app.RunResultProcessor.generateReports(RunResultProcessor.scala:59)
at io.gatling.app.RunResultProcessor.processRunResult(RunResultProcessor.scala:38)
at io.gatling.app.Gatling$.start(Gatling.scala:81)
at io.gatling.app.Gatling$.fromArgs(Gatling.scala:46)
at io.gatling.app.Gatling$.main(Gatling.scala:38)
at io.gatling.app.Gatling.main(Gatling.scala)
So there's some part when it's never accessed or used.
Any bit of help or pointing me in the right direction would be appreciated.
Thank you!
an asLongAs loop condition is evaluated at the start of the loop - so on your first execution the condition will fail due to there being no session value for statuss.
the doWhile loop provides checking at the end of the loop.

Akka Sink never closes

I am uploading a single file to an SFTP server using Alpakka but once the file is uploaded and I have gotten the success response the Sink stays open, how do I drain it?
I started off with this:
val sink = Sftp.toPath(path, settings, false)
val source = Source.single(ByteString(data))
​
source
.viaMat(KillSwitches.single)(Keep.right)
.toMat(sink)(Keep.both).run()
.map(_.wasSuccessful)
But that ends up never leaving the map step.
I tried to add a killswitch, but that seems to have had no effect (neither with shutdown or abort):
val sink = Sftp.toPath(path, settings, false)
val source = Source.single(ByteString(data))
​
val (killswitch, result) = source
.viaMat(KillSwitches.single)(Keep.right)
.toMat(sink)(Keep.both).run()
result.map {
killswitch.shutdown()
_.wasSuccessful
}
Am I doing something fundamentally wrong? I just want one result.
EDIT The settings sent in to toPath:
SftpSettings(InetAddress.getByName(host))
.withCredentials(FtpCredentials.create(amexUsername, amexPassword))
.withStrictHostKeyChecking(false)
By asking you to put Await.result(result, Duration.Inf) at the end I wanted to check the theory expressed by A. Gregoris. Thus it's either
your app exits before Future completes or
(if you app does't exit) function in which you do this discards result
If your app doesn't exit you can try using result.onComplete to do necessary work.
I cannot see your whole code but it seems to me that in the snippet you posted that result value is a Future that is not completing before the end of your program execution and that is because the code in the map is not being executed either.

scala testing akka actors adding values to the db

I am trying to test run semi integration test with actors, that involve persisting to the db. the fooActor is adding kids to the db. but I have to wait till the persistence to the db will be completed using Thread.sleep :
"persist kids" in {
fooActor ! addChildToParent(bar,foo)
expectMsg("done")
fooActor ! addChildToParent(buz,foo)
expectMsg("done")
Thread.sleep(2000)
suggestionActor ! GetChildListForParent(foo)
expectMsg(
Some(List(bar,buz))
)
}
well this works fine, but I hate the fact that I have to use the Tread.sleep(2000). is there a way to minimize this effect ?
To minimize the effect, you could use awaitAssert
awaitAssert {
suggestionActor ! GetChildListForParent(foo)
expectMsg(
Some(List(bar,buz))
)
}
This awaitAssert retries the given block as long as the expected message is not received. It continues retrying until either the block does not throw an exception (in which case the test succeeds) or until the max duration expires (in which case the test fails). The max duration and interval are optional arguments of awaitAssert.

Future map's (waiting) execution context. Stops execution with FixedThreadPool

// 1 fixed thread
implicit val waitingCtx = scala.concurrent.ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1))
// "map" will use waitingCtx
val ss = (1 to 1000).map {n => // if I change it to 10 000 program will be stopped at some point, like locking forever
service1.doServiceStuff(s"service ${n}").map{s =>
service1.doServiceStuff(s"service2 ${n}")
}
}
Each doServiceStuff(name:String) takes 5 seconds. doServiceStuff does not have implicit ex:Execution context as parameter, it uses its own ex context inside and does Future {blocking { .. }} on it.
In the end program prints:
took: 5.775849753 seconds for 1000 x 2 stuffs
If I change 1000 to 10000 in, adding even more tasks : val ss = (1 to 10000) then program stops:
~17 027 lines will be printed (out of 20 000). No "ERROR" message
will be printed. No "took" message will be printed
**And will not be processing any futher.
But if I change exContext to ExecutionContext.fromExecutor(null: Executor) (global one) then in ends in about 10 seconds (but not normally).
~17249 lines printed
ERROR: java.util.concurrent.TimeoutException: Futures timed out after [10 seconds]
took: 10.646309398 seconds
That's the question
: Why with fixed ex-context pool it stops without messaging, but with global ex-context it terminates but with error and messaging?
and sometimes.. it is not reproducable.
UPDATE: I do see "ERROR" and "took" if I increase pool from 1 to N. Does not matter how hight N is - it sill will be the ERROR.
The code is here: https://github.com/Sergey80/scala-samples/tree/master/src/main/scala/concurrency/apptmpl
and here, doManagerStuff2()
I think I have an idea of what's going on. If you squint enough, you'll see that map duty is extremely lightweight: just fire off a new future (because doServiceStuff is a Future). I bet the behavior will change if you switch to flatMap, which will actually flatten the nested future and thus will wait for second doServiceStuff call to complete.
Since you're not flattening out these futures, all your awaits downstream are awaiting on a wrong thing, and you are not catching it because here you're discarding whatever Service returns.
Update
Ok, I misinterpreted your question, although I still think that that nested Future is a bug.
When I try your code with both executors with 10000 task I do get OutOfMemory when creating threads in ForkJoin execution context (i.e. for service tasks), which I'd expect. Did you use any specific memory settings?
With 1000 tasks they both do complete successfully.

Sending a message to a TestProbe() fails sometimes with ActorInitializationException

I've some sporadic test failures and struggle to figure out why. I have a bunch of actors which to the work which I want to test. At the beginning of the test I pass in an actor reference which I get from a TestProbe(). Later on the group of actors do some work and send the result to the given test probe actor reference. Then I check the result with the TestProbe():
class MyCaseSpec extends Spec with ShouldMatchers{
describe("The Thingy"){
it("should work"){
val eventListener = TestProbe()
val myStuffUnderTest = Actor.actorOf(new ComplexActor(eventListener.ref)).start();
myStuffUnderTest ! "Start"
val eventMessage = eventListener.receiveOne(10.seconds).asInstanceOf[SomeEventMessage]
eventMessage.data should be ("Result")
}
}
}
Now once in a while the test fails. And when I look through the stack trace I see that I got a 'ActorInitializationException' when sending a message to the test probe actor. However at no point in time I stop the TestProbe actor.
Here's the exception:
[akka:event-driven:dispatcher:global-11] [LocalActorRef] Actor has not been started, you need to invoke 'actor.start()' before using it
akka.actor.ActorInitializationException: Actor has not been started, you need to invoke 'actor.start()' before using it
[Gamlor-Laptop_c15fdca0-219e-11e1-9579-001b7744104e]
at akka.actor.ScalaActorRef$class.$bang(ActorRef.scala:1399)
at akka.actor.LocalActorRef.$bang(ActorRef.scala:605)
at akka.mobile.client.RemoteMessaging$RemoteMessagingSupervision$$anonfun$receive$1.apply(RemoteMessaging.scala:125)
at akka.mobile.client.RemoteMessaging$RemoteMessagingSupervision$$anonfun$receive$1.apply(RemoteMessaging.scala:121)
at akka.actor.Actor$class.apply(Actor.scala:545)
....
I'm wondering if I'm missing something obvious or am I making a subtle mistake? Or maybe something is really going wrong inside my code and I can't see it?
I'm on Akka 1.2.
Update for Vitors-Comment. At line 125 I send a message to an actor with the !-operator. Now in the test-setup thats the TestProbe actor-reference. And I can't figure out why sometimes the TestProbe actor seems to be stopped.
protected def receive = {
case msg: MaximumNumberOfRestartsWithinTimeRangeReached => {
val lastException = msg.getLastExceptionCausingRestart
faultHandling ! ConnectionError(lastException, messages.toList, self) // < Line 125. The faultHandling is the TestProbe actor
become({
// Change to failure-state behavior
}
// Snip
Anyway, I'm trying to isolate the problem further for the time being. Thanks for any hint / idea.
You are not starting your actor here. I'm not sure why your test is working some of the time. The code above needs to have the following line modified with an .start()
val myStuffUnderTest = Actor.actorOf(new ComplexActor(eventListener.ref)).start();
Ok, almost certainly found the issue =). TestProbes do have a timeout: When nothing happens after 5 seconds, they stop them self.
Now unfortunately the test takes just a little longer than 5 seconds: In that time the test-probe may stop itself all ready, which then causes the test to fail.
Fixing it is easy, increase the timeout on the TestProbe:
val errorHandler = ignoreConnectionMsgProbe()
errorHandler.setTestActorTimeout(20.seconds)