Premises
Rest API in scala / spray
Simple method that always returns OK
I try to achieve an average of 20k requests per second
Both machines (tester & tested) well configured (EC2 dedicated servers, each one with only its API & Gatling, configuration files
sudo sysctl -w net.ipv4.ip_local_port_range="1025 65535"
echo 300000 | sudo tee /proc/sys/fs/nr_open
echo 300000 | sudo tee /proc/sys/fs/file-max, /etc/security/limits.conf, ulimit -a --> 65535)
This is my simple test file scenario, just 1 user:
setUp(scn.inject(constantUsersPerSec(1) during(60 seconds)))
.throttle(
//reachRps(20000) in (60 seconds),
//holdFor(1 minute)
//,
jumpToRps(20000),
holdFor(1 minutes)
)
.protocols(httpConf)
I try to achieve 20k rqs (maximum) in 60sec or jump directly to 20k and maintain there along 1 minute.
This is always my results after executing Gatling script:
Simulation finished
Parsing log file(s)...
Parsing log file(s) done
Generating reports...
================================================================================
---- Global Information --------------------------------------------------------
> request count 60 (OK=60 KO=0 )
> min response time 0 (OK=0 KO=- )
> max response time 2 (OK=2 KO=- )
> mean response time 1 (OK=1 KO=- )
> std deviation 0 (OK=0 KO=- )
> response time 50th percentile 1 (OK=1 KO=- )
> response time 75th percentile 2 (OK=2 KO=- )
> mean requests/sec 1.017 (OK=1.017 KO=- )
---- Response Time Distribution ------------------------------------------------
> t 800 ms t > 1200 ms 0 ( 0%)
> failed 0 ( 0%)
I don't understand what means exactly these results... or, perhaps, I'm not configuring the right scenario for my goal.
I tried with several scenaries:
//setUp(scn.inject(atOnceUsers(20000)).protocols(httpConf))
//setUp(scn.inject(Users(200000).ramp(10)).protocols(httpConf))
//setUp(scn.inject(constantUsersPerSec(20000) during(1 seconds)).protocols(httpConf))
//setUp(scn.inject(constantUsersPerSec(20000) during(1 seconds))).protocols(httpConf)
//setUp(scn.inject(rampUsers(1500) over (60 seconds)))
//setUp(scn.inject(atOnceUsers(50000)))
// .throttle(jumpToRps(50000),
// holdFor(1 minutes))
// .protocols(httpConf)
setUp(scn.inject(constantUsersPerSec(1000) during(30 seconds)))
.throttle(
reachRps(20000) in (30 seconds),
holdFor(1 minute)
//,
//jumpToRps(20000),
//holdFor(1 minutes)
)
.protocols(httpConf)
So, I don't know how to configure my scala test file for, simply, getting a value like that:
> mean requests/sec 20000 (OK=20000 KO=- )
You don't get throttle right. From the documentation:
You still have to inject users at the scenario level. Throttling tries
to ensure a targeted throughput with the given scenarios and their
injection profiles (number of users and duration). It’s a bottleneck,
ie an upper limit. If you don’t provide enough users, you won’t reach
the throttle. If your injection lasts less than the throttle, your
simulation will simply stop when all the users are done. If your
injection lasts longer than the throttle, the simulation will stop at
the end of the throttle.
How can you expect reaching 20000 rps whith only injection 1 user per second?
Related
Right now I'm injecting users like this, which will gradually increase the number of users over the period of 30 minutes:
setUp(
users.inject(rampUsers(300) over ((30) minutes))
).protocols(httpConf)
What I want is during the duration of 1st and 10th minute inject the 300 users, during the duration of 11th and 20th minute inject 300 more users which will be 300 + 300 = 600 and during the duration of 21st and 30th minute inject 300 more users which will be 300 + 300 + 300 = 900. In short, I want to double the number of users after every 10 minutes
By the sounds of it, you want 10 minutes where there are 300 users followed by 10 min where there are 600, and finally another 10 where there are 900.
You could used one of the closed injection profiles to achieve this.
scn.inject(
constantConcurrentUsers(300) during (10 minutes),
constantConcurrentUsers(600) during (10 minutes),
constantConcurrentUsers(900) during (10 minutes)
)
You can simply concatenate those commands:
scn.inject(
rampUsers(300) during (10 minutes),
rampUsers(300) during (10 minutes),
rampUsers(300) during (10 minutes),
)
Like the documentation says:
The definition of the injection profile of users is done with the inject method. This method takes as argument a sequence of injection steps that will be processed sequentially. (https://gatling.io/docs/2.3/general/simulation_setup)
Also, the behavior you described could be achieved by just doing rampUsers(900) during (30 minutes).
I have two excel extracts, one containing the schedules of employees and the other containing the actual times the employees worked. There are multiple schedule blocks in a day and employees can have multiple activity start and stop times in or out of scheduled time.
Schedule:
Actvtivity:
I need to use this data to calculate the sum of actual minutes worked only during the scheduled time with Tableau. In the above images the first schedule event would have 115 minutes worked and the second line would have 92 totaling in 207.
I tried to start easy by only calculating activity entries that fall within both the start and stop of a scheduled block, but tableau either return nothing or the "Many to 1" *
attempt 1:
case attr([#Fields:Date]) when ATTR([ScheduleStopStart].[#fields:excDate])
then
if attr([Start]) >= Min([ScheduleStopStart].[Start])
and attr([Stop]) <= max([ScheduleStopStart].[Stop])
then datediff("minute", attr([Start]), attr([Stop]))
END
END
Attempt 2:
IF ATTR([#Fields:Date]) = ATTR([ScheduleStopStart].[#fields:excDate]) then (
if attr([Start]) >= attr([ScheduleStopStart].[Start])
and attr([Stop]) <= attr([ScheduleStopStart].[Stop])
then DATEDIFF("minute", attr([Start]),attr([Stop]))
else 0
end
)
else
0
END
I am running a simple select statement in my database (connected from a remote machine using putty). I connected to SQLPLUS from the putty and executing the select statement. But getting different response times each time I run the query. Here are my observations.
1) Enabled the trace(10046). "elapsed_time" in the trace file is different for each execution of the query.
2) there is a huge difference in the elapsed time which is displayed on the console, and which is there in trace file... From the Putty, the elapsed time is showing approx. 2-3 secs. Whereas the elapsed time logged in the trace is showing the elapsed time as 1 sec... What is the difference between elapsed time on the putty console and trace file log?
Putty Console output:
select * from WWSH_TEST.T01DATA_LINK
489043 rows selected.
Elapsed: 00:02:57.16
Tracefile output:
select *
from
WWSH_TEST.T01DATA_LINK
call count cpu elapsed disk query current rows
---- ------- ------- --------- -------- -------- ------- ------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 32604 0.38 2.32 10706 42576 0 489043
From the putty console, the elapsed time is showing as 2.57 secs, whereas from the trace file the elapsed time is 2.32 .. Why we see this difference?
Moreover, when I am running this same SQL statement repeatedly, I could see different elapsed times in the trace files (ranging from 2.3 to 2.5 secs) .. What could be the reason for this difference in response time when there is no change in the database at all.
Database Version: 11.2.0.3.0
It looks like the time difference is the "client processing" - basically the time spent by sqlplus formatting the output.
Also, it looks like your array size is 15. Try running with a larger array size, such as 512 or 1024.
The mechanism to set the array size will vary from client to client. In sqlplus:
set arraysize 1024
The fetch time does not include network time, but if you use a level 8 trace
alter session set events '10046 trace name context forever, level 8';
this will given you wait events. The one to look for is SQL*Net message from client, which is basically the time the database is waiting to be asked to do something, such as fetch the next set of rows.
I run an action two times, and the second time takes very little time to run, so I suspect that spark automatically cache some results. But I did find any source.
Im using Spark1.4.
doc = sc.textFile('...')
doc_wc = doc.flatMap(lambda x: re.split('\W', x))\
.filter(lambda x: x != '') \
.map(lambda word: (word, 1)) \
.reduceByKey(lambda x,y: x+y)
%%time
doc_wc.take(5) # first time
# CPU times: user 10.7 ms, sys: 425 µs, total: 11.1 ms
# Wall time: 4.39 s
%%time
doc_wc.take(5) # second time
# CPU times: user 6.13 ms, sys: 276 µs, total: 6.41 ms
# Wall time: 151 ms
From the documentation:
Spark also automatically persists some intermediate data in shuffle operations (e.g. reduceByKey), even without users calling persist. This is done to avoid recomputing the entire input if a node fails during the shuffle. We still recommend users call persist on the resulting RDD if they plan to reuse it.
The underlying filesystem will also be caching access to the disk.
I need to write Gatling scenario that will mimic real users interaction. It's supposed to issue some requests occasionally, e.g. 10 per hour per user (total 20 users).
From what I see in the docs, constantUsersPerSec accepts double but it's rounded while reachRps in throttling deals only with seconds. So, not way to have less than 1 rps.
It is possible to write such scenario using Gatling?
So your scenario seems like "for 2 hours, send a request every 6 minutes" or "at a constant rate of 10 users per hour during 2 hours ...".
Option 1
The constantUsersPerSec is internally rounded to int after multiplying it to the number of seconds of the duration. So the duration should be chosen with respect to the rate, so that the result is greater than 1.
In your case,
def perHour(rate : Double): Double = rate / 3600
constantUsersPerSec(perHour(10)) during(2 hours)
This would result in
10/3600 users * (2 * 60 * 60) seconds = 20 users
Option 2
via injection steps
setUp(
scn.inject(
atOnceUsers(1),
nothingFor(6 minutes),
atOnceUsers(1),
nothingFor(6 minutes),
//... and so forth...
)
)
or produce the injection steps in a second method
def injections(): List[InjectionStep] = List(...)
setUp(scn.inject(injections : _*))