I have a kafka simulation test using Gatling and currently we are able to run with the constantUsersPerSec for a given duration. But we wanted to run this simulation at constantUsersPerSec (say some 1000/sec) and this test should be running forever until somebody kills the test/process.
Currently we have adjusted to use days in place of duration. But is there an option to run a test forever that uses a feeder. Here is the code snippet:
val scn = scenario("Kafka Test Demo")
.feed(ssv(conf.getString("filename"), true).circular)
.exec(kafka("request")
// message to send
.send[String]("${t}"))
setUp(
scn.inject(constantUsersPerSec(conf.getInt("constantUsersPerSec")) during (conf.getInt("run_duration_in_mins") hours)))
.protocols(kafkaConf)
Any suggestions/directions please?
many thanks,
mpha
Is code bellow going to work?
val scn = scenario("Kafka Test Demo")
.feed(ssv(conf.getString("filename"), true)).circular.forever() {
exec(kafka("request")
// message to send
.send[String]("${t}"))
}
Related
I'm building an app that has the following flow:
There is a source of items to process
Each item should be processed by external command (it'll be ffmpeg in the end but for this simple reproducible use case it is just cat to have data be passed through it)
In the end, the output of such external command is saved somewhere (again, for the sake of this example it just saves it to a local text file)
So I'm doing the following operations:
Prepare a source with items
Make an Akka graph that uses Broadcast to fan-out the source items into individual flows
Individual flows uses ProcessBuilder in conjunction with Flow.fromSinkAndSource to build flow out of this external process execution
End the individual flows with a sink that saves the data to a file.
Complete code example:
import akka.actor.ActorSystem
import akka.stream.scaladsl.GraphDSL.Implicits._
import akka.stream.scaladsl._
import akka.stream.ClosedShape
import akka.util.ByteString
import java.io.{BufferedInputStream, BufferedOutputStream}
import java.nio.file.Paths
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, ExecutionContext, Future}
object MyApp extends App {
// When this is changed to something above 15, the graph just stops
val PROCESSES_COUNT = Integer.parseInt(args(0))
println(s"Running with ${PROCESSES_COUNT} processes...")
implicit val system = ActorSystem("MyApp")
implicit val globalContext: ExecutionContext = ExecutionContext.global
def executeCmdOnStream(cmd: String): Flow[ByteString, ByteString, _] = {
val convertProcess = new ProcessBuilder(cmd).start
val pipeIn = new BufferedOutputStream(convertProcess.getOutputStream)
val pipeOut = new BufferedInputStream(convertProcess.getInputStream)
Flow
.fromSinkAndSource(StreamConverters.fromOutputStream(() ⇒ pipeIn), StreamConverters.fromInputStream(() ⇒ pipeOut))
}
val source = Source(1 to 100)
.map(element => {
println(s"--emit: ${element}")
ByteString(element)
})
val sinksList = (1 to PROCESSES_COUNT).map(i => {
Flow[ByteString]
.via(executeCmdOnStream("cat"))
.toMat(FileIO.toPath(Paths.get(s"process-$i.txt")))(Keep.right)
})
val graph = GraphDSL.create(sinksList) { implicit builder => sinks =>
val broadcast = builder.add(Broadcast[ByteString](sinks.size))
source ~> broadcast.in
for (i <- broadcast.outlets.indices) {
broadcast.out(i) ~> sinks(i)
}
ClosedShape
}
Await.result(Future.sequence(RunnableGraph.fromGraph(graph).run()), Duration.Inf)
}
Run this using following command:
sbt "run PROCESSES_COUNT"
i.e.
sbt "run 15"
This all works quite well until I raise the amount of "external processes" (PROCESSES_COUNT in the code). When it's 15 or less, all goes well but when it's 16 or more then the following things happen:
Whole execution just hangs after emitting the first 16 items (this amount of 16 items is Akka's default buffer size AFAIK)
I can see that cat processes are started in the system (all 16 of them)
When I manually kill one of these cat processes in the system, something frees up and processing continues (of course in the result, one file is empty because I killed its processing command)
I checked that this is caused by the external execution for sure (not i.e. limit of Akka Broadcast itself).
I recorded a video showing these two situations (first, 15 items working fine and then 16 items hanging and freed up by killing one process) - link to the video
Both the code and video are in this repo
I'd appreciate any help or suggestions where to look solution for this one.
It is an interesting problem and it looks like that the stream is dead-locking. The increase of threads may be fixing the symptom but not the underlying problem.
The problem is following code
Flow
.fromSinkAndSource(
StreamConverters.fromOutputStream(() => pipeIn),
StreamConverters.fromInputStream(() => pipeOut)
)
Both fromInputStream and fromOutputStream will be using the same default-blocking-io-dispatcher as you correctly noticed. The reason for using a dedicated thread pool is that both perform Java API calls that are blocking the running thread.
Here is a part of a thread stack trace of fromInputStream that shows where blocking is happening.
at java.io.FileInputStream.readBytes(java.base#11.0.13/Native Method)
at java.io.FileInputStream.read(java.base#11.0.13/FileInputStream.java:279)
at java.io.BufferedInputStream.read1(java.base#11.0.13/BufferedInputStream.java:290)
at java.io.BufferedInputStream.read(java.base#11.0.13/BufferedInputStream.java:351)
- locked <merged>(a java.lang.ProcessImpl$ProcessPipeInputStream)
at java.io.BufferedInputStream.read1(java.base#11.0.13/BufferedInputStream.java:290)
at java.io.BufferedInputStream.read(java.base#11.0.13/BufferedInputStream.java:351)
- locked <merged>(a java.io.BufferedInputStream)
at java.io.FilterInputStream.read(java.base#11.0.13/FilterInputStream.java:107)
at akka.stream.impl.io.InputStreamSource$$anon$1.onPull(InputStreamSource.scala:63)
Now, you're running 16 simultaneous Sinks that are connected to a single Source. To support back-pressure, a Source will only produce an element when all Sinks send a pull command.
What happens next is that you have 16 calls to method FileInputStream.readBytes at the same time and they immediately block all threads of default-blocking-io-dispatcher. And there are no threads left for fromOutputStream to write any data from the Source or perform any kind of work. Thus, you have a dead-lock.
The problem can be fixed if you increase the threads in the pool. But this just removes the symptom.
The correct solution is to run fromOutputStream and fromInputStream in two separate thread pools. Here is how you can do it.
Flow
.fromSinkAndSource(
StreamConverters.fromOutputStream(() => pipeIn).async("blocking-1"),
StreamConverters.fromInputStream(() => pipeOut).async("blocking-2")
)
with following config
blocking-1 {
type = "Dispatcher"
executor = "thread-pool-executor"
throughput = 1
thread-pool-executor {
fixed-pool-size = 2
}
}
blocking-2 {
type = "Dispatcher"
executor = "thread-pool-executor"
throughput = 1
thread-pool-executor {
fixed-pool-size = 2
}
}
Because they don't share the pools anymore, both fromOutputStream and fromInputStream can perform their tasks independently.
Also note that I just assigned 2 threads per pool to show that it's not about the thread count but about the pool separation.
I hope this helps to understand akka streams better.
Turns out this was limit on Akka configuration level of blocking IO dispatchers:
So changing that value to something bigger than the amount of streams fixed the issue:
akka.actor.default-blocking-io-dispatcher.thread-pool-executor.fixed-pool-size = 50
I am uploading a single file to an SFTP server using Alpakka but once the file is uploaded and I have gotten the success response the Sink stays open, how do I drain it?
I started off with this:
val sink = Sftp.toPath(path, settings, false)
val source = Source.single(ByteString(data))
source
.viaMat(KillSwitches.single)(Keep.right)
.toMat(sink)(Keep.both).run()
.map(_.wasSuccessful)
But that ends up never leaving the map step.
I tried to add a killswitch, but that seems to have had no effect (neither with shutdown or abort):
val sink = Sftp.toPath(path, settings, false)
val source = Source.single(ByteString(data))
val (killswitch, result) = source
.viaMat(KillSwitches.single)(Keep.right)
.toMat(sink)(Keep.both).run()
result.map {
killswitch.shutdown()
_.wasSuccessful
}
Am I doing something fundamentally wrong? I just want one result.
EDIT The settings sent in to toPath:
SftpSettings(InetAddress.getByName(host))
.withCredentials(FtpCredentials.create(amexUsername, amexPassword))
.withStrictHostKeyChecking(false)
By asking you to put Await.result(result, Duration.Inf) at the end I wanted to check the theory expressed by A. Gregoris. Thus it's either
your app exits before Future completes or
(if you app does't exit) function in which you do this discards result
If your app doesn't exit you can try using result.onComplete to do necessary work.
I cannot see your whole code but it seems to me that in the snippet you posted that result value is a Future that is not completing before the end of your program execution and that is because the code in the map is not being executed either.
I have a play(2.3.0) application that does some database lookups. When there are more than 6 users the application runs into performance problems.
I have narrowed down the problem to a controller with an action that does a sleep of 4 seconds.
A test client calls this action every 500 ms. I can see the the first 6 requests are processesed, and it stops a few seconds(until the 4 seconds sleep have passed) and reads the next 6.
Also: when I open 7 browser windows the 7th will not load(waits for connection).
Looking at the documentation it looks like my problem is blocking io and using the highly synchronous profile should solve my problem.
Therefore I added this profile to my application.conf but nothing changes.
my application.conf looks like this
application.context=/appname/
# Secret key
# ~~~~~
# The secret key is used to secure cryptographics functions.
# If you deploy your application to several instances be sure to use the same key!
application.secret="xxxxx"
play {
akka {
akka.loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = WARNING
actor {
default-dispatcher = {
fork-join-executor {
parallelism-min = 300
parallelism-max = 300
}
}
}
}
}
and the action
def performancetestSleep() = Action{ request => {
Thread.sleep(4000)
Ok("hmmm good sleep")
}}
It seems to me the threadpool configuration is ignored. What am I missing here?
What you need for this is really just one thread which handles the 4 second delay - a scheduler. Spawning that many threads defeats the whole point of the architecture that Play has, IMHO. You could then use the scheduler to create a Future[Result] which you'd feed into an Action.async block.
Now, you don't really need to implement your own scheduler since Play depends on Akka for its concurrency; and Akka has a scheduler which will do the job.
import scala.concurrent.{Promise}
import scala.concurrent.duration._
import play.libs.Akka
val system = Akka.system()
def delayedResponse = Action.async {
import system.dispatcher
val promise = Promise[Result]
system.scheduler.scheduleOnce(4000 milliseconds) {
promise.success(Ok("Sorry for the wait!"))
}
promise.future
}
I used
activator run
to start the server, that does not seem to pick up the threadpool profile. Using
activator start
does, and now the profile seems to be used. I now need to test if this solves my problem. Will also have a look at the async call.
I am working on a Gatling simulation. For the life of me, I cannot get my code to reach 10000 requests per second. I have read the documentation and I keep messing with different methods and whatnot but my requests per second seems capped at 5000 requests per second. I have attached my current iteration of my code. The URL and path information is blurred out. Assume that I have no issue with the HTTP part of my simulation.
package computerdatabase
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
//import assertions._
class userSimulation extends Simulation {
object Query {
val feeder = csv("firstfileSHUF.txt").random
val query = repeat(2000) {
feed(feeder).
exec(http("user")
.get("/path/path/" + "${userID}" + "?fullData=true"))
}
}
val baseUrl = "http:URL:7777"
val httpConf = http
.baseURL(baseUrl) // Here is the root for all relative URLs
val scn = scenario("user") // A scenario is a chain of requests and pauses
.exec(Query.query)
setUp(scn.inject(rampUsers(1500) over (60 seconds)))
.throttle(reachRps(10000) in (2 minute),
holdFor(3 minutes))
.protocols(httpConf)
}
Additionally, I would like to set the maximum threshold for a timeout to be 100ms. I have tried to do this with assertions and also editing the configuration files but it never seems to show up during the tests or in my reports. How can I set a request to KO if the request took longer than 100ms? Thank you for your help with this matter!
I ended up figuring this out. My code above is correct and I know understand what Stephane, one of the main contributors to Gatling was explaining. The server at the time simply could not handle my RPS threshold. It was an upper bound that was unreachable. After making changes to the server, we could handle this sort of latency. Additionally, I found a way to timeout at 100ms in the configuration file. Specifically, requestTimeout = 100 will cause the timeout behavior I was looking for.
I've some sporadic test failures and struggle to figure out why. I have a bunch of actors which to the work which I want to test. At the beginning of the test I pass in an actor reference which I get from a TestProbe(). Later on the group of actors do some work and send the result to the given test probe actor reference. Then I check the result with the TestProbe():
class MyCaseSpec extends Spec with ShouldMatchers{
describe("The Thingy"){
it("should work"){
val eventListener = TestProbe()
val myStuffUnderTest = Actor.actorOf(new ComplexActor(eventListener.ref)).start();
myStuffUnderTest ! "Start"
val eventMessage = eventListener.receiveOne(10.seconds).asInstanceOf[SomeEventMessage]
eventMessage.data should be ("Result")
}
}
}
Now once in a while the test fails. And when I look through the stack trace I see that I got a 'ActorInitializationException' when sending a message to the test probe actor. However at no point in time I stop the TestProbe actor.
Here's the exception:
[akka:event-driven:dispatcher:global-11] [LocalActorRef] Actor has not been started, you need to invoke 'actor.start()' before using it
akka.actor.ActorInitializationException: Actor has not been started, you need to invoke 'actor.start()' before using it
[Gamlor-Laptop_c15fdca0-219e-11e1-9579-001b7744104e]
at akka.actor.ScalaActorRef$class.$bang(ActorRef.scala:1399)
at akka.actor.LocalActorRef.$bang(ActorRef.scala:605)
at akka.mobile.client.RemoteMessaging$RemoteMessagingSupervision$$anonfun$receive$1.apply(RemoteMessaging.scala:125)
at akka.mobile.client.RemoteMessaging$RemoteMessagingSupervision$$anonfun$receive$1.apply(RemoteMessaging.scala:121)
at akka.actor.Actor$class.apply(Actor.scala:545)
....
I'm wondering if I'm missing something obvious or am I making a subtle mistake? Or maybe something is really going wrong inside my code and I can't see it?
I'm on Akka 1.2.
Update for Vitors-Comment. At line 125 I send a message to an actor with the !-operator. Now in the test-setup thats the TestProbe actor-reference. And I can't figure out why sometimes the TestProbe actor seems to be stopped.
protected def receive = {
case msg: MaximumNumberOfRestartsWithinTimeRangeReached => {
val lastException = msg.getLastExceptionCausingRestart
faultHandling ! ConnectionError(lastException, messages.toList, self) // < Line 125. The faultHandling is the TestProbe actor
become({
// Change to failure-state behavior
}
// Snip
Anyway, I'm trying to isolate the problem further for the time being. Thanks for any hint / idea.
You are not starting your actor here. I'm not sure why your test is working some of the time. The code above needs to have the following line modified with an .start()
val myStuffUnderTest = Actor.actorOf(new ComplexActor(eventListener.ref)).start();
Ok, almost certainly found the issue =). TestProbes do have a timeout: When nothing happens after 5 seconds, they stop them self.
Now unfortunately the test takes just a little longer than 5 seconds: In that time the test-probe may stop itself all ready, which then causes the test to fail.
Fixing it is easy, increase the timeout on the TestProbe:
val errorHandler = ignoreConnectionMsgProbe()
errorHandler.setTestActorTimeout(20.seconds)