For example- I need to check that for 1000 users it responds in 3 seconds.
Is the number of users and response times configurable?
This answer targets Gatling 2.
You can set the target number of users by configuring the "injection profile" of your simulation:
setUp(scn.inject(atOnceUsers(1000)) // To start all users at the same time
setUp(scn.inject(rampUsers(1000) over (30 seconds) // To start gradually, over 30 seconds
For more information, please refer to the injection DSL documentation
If you want to check that all your users responds in less than 3 seconds, one way to ensure this is Gatling's assertions:
setUp(...).assertions(global.responseTime.max.lessThan(3000))
If this assertion fails, meaning at least one user responded in more than 3 seconds, Gatling will clearly indicate the failure after your simulation is completed.
Related
I am doing load test for an api which average response time is 5 sec
in my script i setup constantUserPerSecond 2 and duration 150 second
.inject(constantUsersPerSec(2) during (150 seconds)),
will it generate 2 request per second ? or less , because of 1 request will take 5 second to complete ?
constantUsersPerSec(2) will start a new user executing the scenario every .5 seconds or so. For this sort of injection profile gatling doesn't take into consideration how long it takes for a request to complete.
I like Locust, but I'm having a problem interpreting the results.
e.g. my use case is that I have a petition site. I expect 10,000 people to sign the petition over a 12 hour period.
I've written a locust file that simulates user behaviour:
Some users load but don't sign petition
Some users load and submit invalid data
Some users (hopefully) successfully submit.
In real life the user now goes away (because the petition is an API not a main website).
Locust shows me things like:
with 50 concurrent users the median time is 11s
with 100 concurent users the median time is 20s
But as one "Locust" just repeats the tasks over and over, it's not really like one user. If I set it up with a swarm of 1 user, then that still represents many real world users, over a period of time; e.g. in 1 minute it might do the task 5 times: that would be 5 users.
Is there a way I can interpret the data ("this means we can handle N people/hour"), or some way I can see how many "tasks" get run per second or minute etc. (ie locust gives me requests per second but not tasks)
Tasks dont really exist on the logging level in locust.
If you want, you could log your own fake samples, and use that as your task counter. This has an unfortunate side effect of inflating your request rate, but it should not impact things like average response times.
Like this:
from locust.events import request_success
...
#task(1)
def mytask(self):
# do your normal requests
request_success.fire(request_type="task", name="completed", response_time=None, response_length=0)
Here's the hacky way that I've got somewhere. I'm not happy with it and would love to hear some other answers.
Create class variables on my HttpLocust (WebsiteUser) class:
WebsiteUser.successfulTasks = 0
Then on the UserBehaviour taskset:
#task(1)
def theTaskThatIsConsideredSuccessful(self):
WebsiteUser.successfulTasks += 1
# ...do the work...
# This runs once regardless how many 'locusts'/users hatch
def setup(self):
WebsiteUser.start_time = time.time();
WebsiteUser.successfulTasks = 0
# This runs for every user when test is stopped.
# I could not find another method that did this (tried various combos)
# It doesn't matter much, you just get N copies of the result!
def on_stop(self):
took = time.time() - WebsiteUser.start_time
total = WebsiteUser.successfulTasks
avg = took/total
hr = 60*60/avg
print("{} successful\nAverage: {}s/success\n{} successful signatures per hour".format(total, avg, hr)
And then set a zero wait_time and run till it settles (or failures emerge) and then stop the test with the stop button in the web UI.
Output is like
188 successful
0.2738157498075607s/success
13147.527132862522 successful signatures per hour
I think this therefore gives me the max conceivable throughput that the server can cope with (determined by changing the No. users hatched until failures emerge, or until the average response time becomes unbearable).
Obviously real users would have pauses, but that makes it harder to test the maximums.
Drawbacks
Can't use distributed Locust instances
Messy; also can't 'reset' - have to quit the process and restart for another test.
I'm using Entity Framework Core 2.2 and i decided to follow some blog sugestion and enable retry on failure:
services.AddDbContext<MyDbContext>( options =>
options.UseSqlServer(Configurations["ConnectionString"]),
sqlServerOptionsAction: sqlOptions =>
{
sqlOptions.EnableRetryOnFailure(
maxRetryCount: 10,
maxRetryDelay: TimeSpan.FromSeconds(5),
errorNumbersToAdd: null);
});
My question is what is the maxRetryDelay argument for?
I would expect it to be the delay time between retries, but the name implies its the maximum time, does that mean i can do my 10 retries 1 second apart and not 5 seconds apart as i desire?
The delay between retries is randomized up to the value specified by maxRetryDelay.
This is done to avoid multiple retries occuring at the same time and overwhelming a server. Imagine for example 10K requests to a web service failing due to a network issue and retrying at the same time after 15 seconds. The database server would get a sudden wave of 15K queries.
By randomizing the delay, retries are spread across time and client.
The delay for each retry is calculated by ExecutionStragegy.GetNextDelay. The source shows it's a random exponential backoff.
The default SqlServerRetryingExecutionStrategy uses that implementation. A custom retry strategy could use a different implementation
So, I am using JMeter's throughput shaping timer to test the performance of our REST Server. I noticed a few things i did not expect.
First of all my setup details :
1)JMeter Version : 3.0 r1743807
2)JMX file : DropBox Link
Now , my questions :
1)The throughput shaping timer is configured to run for 60 seconds(100rps - 30 secs, 200 rps - next 30 secs). But the actual test runs only for 3 seconds as shown below. Why?
2) As per the plan the number of requests per second should go from 100 - 200. But here it seems to decrease , as in above.
3)As per this plugin's documentation , the number of thread groups = desired requests per second * server response time / 1000 . Is it because how this plugin internally works or is it a simple logic i am missing?
The issue is with the Thread Group settings.
You only one have 1 iteration and ramp up 300 users in 1 second. So if Jmeter can send all the 300 requests and get the response, JMeter will finish the test immediately. Those timer settings will apply only if the test is running.
If you need the test to run for some duration (say 60 seconds), then set the loop count to forever & duration to 60
I am trying to implement a poker server. An http server forwards data packets to the backend servers which handle the state of all the poker hands. In any given hand the player to act gets 10 seconds to act (bet,fold,call,raise,etc.). If there is no response within 10 seconds the server automatically folds for them. To check that 10 seconds has passed an event list of when actions must be received is maintained. It is a priority queue ordered by time and each poker hand currently being played has an entry in the priority queue.
Consider the following scenario since the last action 9.99 seconds pass before the next action arrives at the http server. By the time the action is forwarded to the backend servers extra time passes so now a total of 10.1 seconds have passed. The backend servers will have declared the hand folded, but I would like the action to be processed since technically it arrived at the http server after 9.99 seconds. Now one solution would be to have the backends wait some extra time before declaring a hand folded to see if an action timestamped at 9.99 seconds comes. But that would result in delaying when the next person in the hand gets to act.
The goals I would like are
Handle actions reaching the http server at 9.99 seconds instead of folding their hand.
Aggressively minimize delay resulting from having to do idle waiting to "solve" problem mentioned in bullet point 1.
What are the various solutions? To experts in distributed systems is there known literature on what the trade offs are to various solutions. I would like to know the various solutions deemed acceptable by distributed systems literature. Not just various ad hocs solution.
Maybe on the server side when client request arrives you could take the timestamp?
So you would take "start" and "stop" timestamps, to measure exactly 9.9s?