Is there a way to force a gatling test to consider connections that have been active longer than a certain duration to have failed?
For instance, I have a test that will create 400 users/second for 60 seconds. However I am having the test "hang" indefinitely.
================================================================================
2016-04-13 08:08:25 200s elapsed
---- Full Chain test -----------------------------------------------------------
[##############################################################------------] 84%
waiting: 0 / active: 3728 / done:20362
---- Requests ------------------------------------------------------------------
> Global (OK=20362 KO=0 )
================================================================================
As you can see, the 60 second test, +/- a few seconds for the final requests to complete, has gone on for 200 seconds (this is before killing it). The "active" number has remained at 3728 since the 65 second mark.
This duration goes against all the timeouts I can see in the gatling docs, and setting my own timeouts doesn't appear to do anything. Here's my reference.conf
gatling {
http {
ahc {
requestTimeout=7000
maxRetry=0
sslSessionTimeout=7000
}
}
data {
noActivityTimeout=5
}
}
Has anyone figured out a way to get around this issue?
Please upgrade to Gatling 2.2.0 that we've just released. There's a good chance it fixes your bug.
Related
We have SumoLogic alert that happens if more than 10 errors logged in 60 min.
I prefer to have something like:
if there is a spike and all the errors happen in e.g. 1 minute ( consider as issue has been auto resolved ) do not generate alert.
How can I set such sumoLogic query?
Variances of the requirements :
Logs have clientIp field, and if all errors are reported for the same client, do not generate alert( problem with particular client, not with application)
if more than 10 errors logged in 60 min, send an alert, unless the errors are of type A, but if there are more than 100 errors of type A, send the alert.( log errors of type A are acceptable, unless the number is too big)
if more than 10 errors logged in 60 min, send an alert Only if the last error happened less than 30 min ago(otherwise consider as auto-fixed)
I am not fully sure how is your data shaped, but...
if there is a spike and all the errors happen in e.g. 1 minute ( consider as issue has been auto resolved ) do not generate alert.
This you can solve by aggregating:
| timeslice 1m
| count by _timeslice
| where _count > 1
or similar.
if all errors are reported for the same client, do not generate alert
It sounds like:
| count by _timeslice, clientIp
would do the job.
if more than 10 errors logged in 60 min, send an alert, unless the errors are of type A, but if there are more than 100 errors of type A,
Rough sketch of the query clause would be:
| if(something, 1, 0) as is_of_type_A
| count by is_of_type_A, ...
| where (is_of_type_A = 1 and _count > 100)
OR (is_of_type_A = 0 and _count > 10)
Disclaimer: I am currently employed by Sumo Logic.
I am looking to create three scenarios:
The first scenario will run a bunch of GET requests for 30s
The second and third scenarios will run in parallel and wait until the first is finished.
I want the requests from the first scenario to be excluded from the report.
I have the basic outline of what I want to achieve but not seeing expected results:
val myFeeder = csv("somefile.csv")
val scenario1 = scenario("Get stuff")
.feed(myFeeder)
.during(30 seconds) {
exec(
http("getStuff(${csv_colName})").get("/someEndpoint/${csv_colName}")
)
}
val scenario2 = ...
val scenario3 = ...
setUp(
scenario1.inject(
constantUsersPerSec(20) during (30 seconds)
).protocols(firstProtocaol),
scenario2.inject(
nothingFor(30 seconds), //wait 30s
...
).protocols(secondProt)
scenario3.inject(
nothingFor(30 seconds), //wait 30s
...
).protocols(thirdProt)
)
I am seeing the first scenario being run throughout the entire test. It doesn't stop after the 30s?
For the first scenario I would like to cycle through the CSV file and perform a request for each line. Perhaps 5-10 requests per second, how do I achieve that?
I would also like it to stop after the 30s and then run the other two in parallel. Hence the nothingFor in last two scenarios above.
Also how do I exclude from report, is it possible?
You are likely not getting the expected results due to the combination of settings between your injection profile and your "Get Stuff" scenario.
constantUsersPerSec(20) during (30 seconds)
will start 20 users on scenario "Get Stuff" every second for 30 seconds. So even during the 30th second, 20 users will START "Get Stuff". The injection pofile only controls when a user starts, not how long they are active for. So when a user executes the "Get Stuff" scenario, they make the 'get' request repeatedly over the course of 30 seconds due to the .during loop.
So at the very least, you will have users executing "Get Stuff" for 60 seconds - well into the execution of your other scenarios. Depending on the execution time for you getStuff call, it may be even longer.
To avoid this, you could work out exactly how long you want the "Get Stuff" scenario to run, set that in the injection profile and have no looping in the scenario. Alternatively, you could just set your 'nothingFor' values to be >60s.
To exclude the Get Stuff calls from reports, you can add silencing to the protocol definition (assuming it's not shared with your other requests). More details at https://gatling.io/docs/3.2/http/http_protocol/#silencing
I am checking a scala code for gatling where they inject transactions for the period of 20 seconds.
/*TPS = Transaction Per Second */
val minTps = Integer.parseInt(System.getProperty("minTps", "1"))
val maxTps = Integer.parseInt(System.getProperty("maxTps", "5"))
var rampUsersDurationInMinutes =Integer.parseInt(System.getProperty("rampUsersDurationInMinutes", "20"))
setUp(scn.inject(
rampUsersPerSec(minTps) to maxTps during (rampUsersDurationInMinutes seconds)).protocols(tcilProtocol))
The same question was asked What does rampUsersPerSec function really do? but never answered. I think that ideally the the graph should be looking like that.
could you please confirm if I correctly understood
rampUsersPerSec?
block (ramp) 1 = 4 users +1
block (ramp) 2 = 12 users +2
block (ramp) 3 = 24 users +3
block (ramp) 4 = 40 users +4
block (ramp) 5 = 60 users +5
The results show that the requests count is indeed 60. Is my calculation correct?
---- Global Information --------------------------------------------------------
> request count 60 (OK=38 KO=22 )
> min response time 2569 (OK=2569 KO=60080 )
> max response time 61980 (OK=61980 KO=61770 )
> mean response time 42888 (OK=32411 KO=60985 )
> std deviation 20365 (OK=18850 KO=505 )
> response time 50th percentile 51666 (OK=32143 KO=61026 )
> response time 75th percentile 60903 (OK=48508 KO=61371 )
> response time 95th percentile 61775 (OK=61886 KO=61725 )
> response time 99th percentile 61974 (OK=61976 KO=61762 )
> mean requests/sec 0.741 (OK=0.469 KO=0.272 )
---- Response Time Distribution ------------------------------------------------
rampUsersPerSec is an open workload model injection where you specify the rate at which users start the scenario. The gatling documentation says that this injection profile
Injects users from starting rate to target rate, defined in users per second, during a given duration. Users will be injected at regular intervals
So while I'm not sure that the example you provide is precisely correct in that gatling is using a second as the 'regular interval' (it might be a smoother model), you are more or less correct. You specify a starting rate and a final rate, and gatling works out all the intermediate injection rates for your duration.
Note that this says nothing about the number of concurrent users your simulation will generate - that is a function of the arrival rate (which you control) and the execution time (which you do not)
I'm a Performance QC engineer, so far i used Visual Studio Ultimate to run load test bug now I'm going to change to gatling. So I'm a newbie on gatling and scala.
I'm defining the simulation with step-load scenario here:
Initial: 5 user
Maximum user count: 100 users
Step duration: 10 seconds
Step user count: 5 users
Duration: 10 minutes
Meaning: start with 5 users > after 10 seconds increase 5 users: repeat until maximum 100 user and run the test in 10 minutes.
I tried some code and other injects but the result is not as expected:
splitUsers(100)
into(rampUsers(5)
over(10 seconds))
separatedBy(10 minutes)
Could you please help me to simulate the step load on gatling?
define the User injection part in setUp something like this
setUp(
scn.inject(
atOnceUsers(5), //Initial: 5 user
nothingFor(10 seconds), //A pause to uniform the step load
splitUsers(100) into atOnceUsers(5) separatedBy(10 seconds) //max user,split time,number of user
).protocols(httpConf))
the duration you can define just by using during function over scenario. Hope it helps
Can you be more specific about the result not being as expected?
According to the documentation your situation should be:
splitUsers(100) into(rampUsers(5) over(10 seconds)) separatedBy atOnceUsers(5)
If test duration is the target then have a look at Throttling in the Gatling documentation.
Has anyone ever created a successful Spock test against an f5 dropped connection?
In my f5 rule, if a situation is satisfied - say a bad cookie, I drop the connection
if { [HTTP::cookie exists "badCookie"] } {
if { not ([HTTP::cookie "badCookie"] matches_regex {^([A-Z0-9_\s]+)$}) } {
drop
}
}
Testing this manually, in a browser, results in a slow but eventual timeout, time limit depending on the browser. But rather than manual tests for each of the f5 rules, I'd like to instead incorporate my tests into our Spock functional test library.
Using Spock, #Timeout() or #Timeout(value=5) just ends up doing a never ending increase in the timeout like:
[spock.lang.Timeout] Method 'abc' has not yet returned - interrupting. Next try in 0.50 seconds.
[spock.lang.Timeout] Method 'abc' has not yet returned - interrupting. Next try in 1.00 seconds.
[spock.lang.Timeout] Method 'abc' has not yet returned - interrupting. Next try in 2.00 seconds.
[spock.lang.Timeout] Method 'abc' has not yet returned - interrupting. Next try in 4.00 seconds.
[spock.lang.Timeout] Method 'abc' has not yet returned - interrupting. Next try in 8.00 seconds.
[spock.lang.Timeout] Method 'abc' has not yet returned - interrupting. Next try in 16.00 seconds.
Using the waitFor method approach in http://fbflex.wordpress.com/2010/08/25/geb-and-grails-tips-tricks-and-gotchas/ or https://github.com/hexacta/weet/blob/master/weet/src/groovy/com/hexacta/weet/pages/AjaxPage.groovy does not close out the method using a 5 second specification either.
An example of the code using each of those approaches (timeout class, timeout method, and waitFor) is at https://gist.github.com/ledlogic/b152370b95e971b3992f
My question is has anyone found a way to successfully run a Spock test to verify f5 rules are dropping connections?
For me using the #ThreadInterrupt annotation alongside the #Timeout annotation worked:
#ThreadInterrupt
#Timeout(value = 100, unit = MILLISECONDS)
def 'timeout test'() {
expect:
while(1) {true}
}
You'll find the full documentation here: http://docs.groovy-lang.org/docs/next/html/documentation/#GroovyConsole-Interrupt
However, this may not be sufficient to interrupt a script: clicking
the button will interrupt the execution thread, but if your code
doesn’t handle the interrupt flag, the script is likely to keep
running without you being able to effectively stop it. To avoid that,
you have to make sure that the Script > Allow interruption menu item
is flagged. This will automatically apply an AST transformation to
your script which will take care of checking the interrupt flag
(#ThreadInterrupt). This way, you guarantee that the script can be
interrupted even if you don’t explicitly handle interruption, at the
cost of extra execution time.