I have simple Locust script with one task with http request.
I want to run this task 100 times on 10 users and than stop run script.
Is there any simple way to do it. I know --run-time parameter but it only stop after the specified amount of time
Below my script:
from locust import HttpUser, task, between
class QuickstartUser(HttpUser):
wait_time = between(1, 2)
host = "https://allegro.pl"
#task(1)
def getHome(self):
self.client.get("/dzial/dom-i-ogrod", name = "Get Home and Garden")
Another option, provided by locust-plugins, is the -i parameter: https://github.com/SvenskaSpel/locust-plugins#command-line-options
It should be a little more reliable, as it explicitly calls runner.quit()
If you are not running distributed you can have a global counter and increment it in the task and once it reaches the desired count you can stop the runner, like :
from locust import HttpUser, task, between
counter=0
class QuickstartUser(HttpUser):
wait_time = between(1, 2)
host = "https://allegro.pl"
#task(1)
def getHome(self):
if counter == 100:
self.environment.runner.stop()
self.client.get("/dzial/dom-i-ogrod", name = "Get Home and Garden")
counter = counter + 1`
and if you are running distributed it is best you use some external counter to keep track of requests like redis or something like that.
Related
from locust import SequentialTaskSet, HttpUser, constant, task
import locust_plugins
class MySeqTask(SequentialTaskSet):
#task
def get_status(self):
self.client.get("/200")
print("Status of 200")
#task
def get_100_status(self):
self.client.get("/100")
print("Status of 100")
class MyLoadTest(HttpUser):
host = "https://http.cat"
tasks = [MySeqTask]
wait_time = constant(1)
Examples for locust-plugins command line options can be found here:
https://github.com/SvenskaSpel/locust-plugins/blob/master/examples/cmd_line_examples.sh
locust -u 5 -t 60 --headless -i 10
# Stop locust after 10 task iterations (this is an upper bound, so you can be sure no more than 10 of iterations will be done)
# Note that in a distributed run the parameter needs to be set on the workers, it is (currently) not distributed from master to worker.
You will run your locust file the same way as normal but add -i to each worker you run. It sounds to me like since it's per worker, you'll need to pre-calculate how many you want each worker to run. So if you have 10 workers and you want to stop after a total of 10000 task iterations, you'd probably do -i 1000 on each worker.
I'm running into a problem like this. I'm currently using pytest to run test cases, and reducing execution time using xdist to run tests in parallel and publishing tests results to TestRail. The issue is when using xdist, pytest-testrail plugin creates Test-Run for each xdist workers and then publishes test cases like Untested.
I tried this hook pytest_terminal_summary to prevent pytest_sessionfinish plugin hook from being call multiple times.
I expect only one test run is created, but still multiple test runs are created.
I ran in to the same problem, but found a kind of workaround with duct tape.
I found that all results are collecting properly in test run, if we run the tests with --tr-run-id key.
If you are using jenkins jobs to automate processes, you can do following:
1) create testrun using testrail API
2) get ID of this test run
3) run the tests with --tr-run-id=$TEST_RUN_ID
I used these docs:
http://docs.gurock.com/testrail-api2/bindings-python
http://docs.gurock.com/testrail-api2/reference-runs
from testrail import *
import sys
client = APIClient('URL')
client.user = 'login'
client.password = 'password'
result = client.send_post('add_run/1', {"name": sys.argv[1], "assignedto_id": 1}).get("id")
print(result)
then in jenkins shell
RUN_ID=`python3 testrail_run.py $BUILD_TAG`
and then
python3 -m pytest -n 3 --testrail --tr-run-id=$RUN_ID --tr-config=testrail.cfg ...
I have setup celery + rabbitmq for on a 3 cluster machine. I have also created a task which generates a regular expression based on data from the file and uses the information to parse text. However, I would like that the process of reading the file is done only once per worker spawn and not on every execution of as task.
from celery import Celery
celery = Celery('tasks', broker='amqp://localhost//')
import re
#celery.task
def add(x, y):
return x + y
def get_regular_expression():
with open("text") as fp:
data = fp.readlines()
str_re = "|".join([x.split()[2] for x in data ])
return str_re
#celery.task
def analyse_json(tw):
str_re = get_regular_expression()
re.match(str_re,tw.text)
In the above code, I would like to open the file and read the output into the string only once per worker, and then the task analyse_json should just use the string.
Any help will be appreciated,
thanks,
Amit
Put the call to get_regular_expression at the module level:
str_re = get_regular_expression()
#celery.task
def analyse_json(tw):
re.match(str_re, tw.text)
It will only be called once, when the module is first imported.
Additionally, if you must have only one instance of your worker running at a time (for example CUDA), you have to use the -P solo option:
celery worker --pool solo
Works with celery 4.4.2.
I think what I want is straight forward.
Python script to restart my Raspberry Pi after 23 hours and 59 minutes. The reason I am trying to do this, instead of set times with a cron job, is the Pi has no onboard battery for a clock so I don't care what the time is (if connected to internet, it will source current time), just a count down of 23 hours and 59 minutes from the script starting.
This is as far as I have got;
def restart():
SendEmail = SendEmail "SYSTEM RESTART", "ncam.py auto restart initiated" msg['Subject'], body)
command = "/usr/bin/sudo /sbin/shutdown -r now"
process = subprocess.Popen(command.split(), stdout=subprocess.PIPE)
output = process.communicate()[0]
Also I want to send an email to myself with the set parameters as above.
You are going to want some variant of this:
import time
import os
currentTime = time.time()
tomorrow = currentTime + 86340000
while time.time() < tomorrow:
do.yourCode()
os.system("poweroff")
Put something like that in your function and it will do what you want.
You should probably change to
while time.time() < tomorrow
to avoid any potential "miss" of the exact millisecond to match.
You can simply restart raspberry with 'sudo reboot' command.
Just put this command inside a python code and run it as system command. For example this code count down from 1 to 10 before restart:
import time
import os
for i in range(1,10):
print 'hello',i
#Do your code here
time.sleep(1)
os.system("sudo reboot")
Use this method to countdown whatever time and restart pi.
I have distinct Rabbit queues each dedicated to a special kind of order processing:
# tasks.py
#celery.task
def process_order_for_product_x(order_id):
pass # elided ...
#celery.task
def process_order_for_product_y(order_id):
pass # elided ...
# settings.py
CELERY_QUEUES = {
"black_hole": {
"binding_key": "black_hole",
"queue_arguments": {"x-ha-policy": "all"}
},
"product_x": {
"binding_key": "product_x",
"queue_arguments": {"x-ha-policy": "all"}
},
"product_y": {
"binding_key": "product_y",
"queue_arguments": {"x-ha-policy": "all"}
},
We have a policy of enforcing explicit routing by setting CELERY_DEFAULT_QUEUE = 'black_hole' and then never consuming from black_hole.
Each of these tasks may use celery's canvas primitives, like so:
# tasks.py
#celery.task
def process_order_for_product_x(order_id):
# These can run in parallel
stage_1_group = group(do_something.si(order_id),
do_something_else.si(order_id))
# These can run in parallel
another_group = group(do_something_at_end.si(order_id),
do_something_else_at_end.si(order_id))
# These run in a linear sequence
process_task = chain(
stage_1_group,
do_something_dependent_on_stage_1.si(order_id),
another_group)
process_task.apply_async()
Supposing I want specific uses of celery.group, celery.chord, celery.chord_unlock, and other canvas tasks to flow through the queue for its corresponding product, rather than getting trapped in a black_hole, is there a way to invoke each particular canvas task with either a custom task name or custom routing_key?
For reasons I won't go into I would prefer to not send all celery.* tasks to a catch-all celery_canvas queue, which is what I am doing in the meantime.
This method allows you to route Celery canvas tasks to the queue of a callback task.
It is possible to specify a custom class-based task router for Celery as described here.
Let's focus on the celery.chord_unlock task. Its signature is defined here.
def unlock_chord(self, group_id, callback, ...):
The second positional argument is the signature of the chord callback task.
Task signatures in Celery are basically dicts, so that gives us an opportunity to access task options, including the task queue name.
Here is an example:
class CeleryRouter(object):
def route_for_task(self, task, args=None, kwargs=None):
if task == 'celery.chord_unlock':
callback_signature = args[1]
options = callback_signature.get('options')
if options:
queue = options.get('queue')
if queue:
return {'queue': queue}
Add it to the Celery config:
CELERY_ROUTES = (CeleryRouter(),
I'm currently using Celery in my project. For some scenarios I need task to chain though different queues:
chain(get_staff.s(url), save_staff.s(dt, partner_id, url))()
Those two functions declared like so:
#task(queue='celery_gevent')
def get_staff(source_url):
#task # send to default queue
def save_staff(suggests, dt, partner, url):
btw, celery_gevent is handled by worker with gevent pool to make http requests.
This example, how you can specify queue implicitly. Also you can explicitly put task in a different queue by specifying additional params, like so:
In [1]: add.apply_async([4,5])
Out[1]: <AsyncResult: bda3dedd-c2c4-44db-be8e-6a97e718f8b0>
$ sudo rabbitmqctl list_queues
Listing queues ...
celery 1
...done.
In [2]: add.apply_async([4,5], queue='your_product')
Out[2]: <AsyncResult: 934f6161-298b-468b-9716-3da6fae58fa5>
$ sudo rabbitmqctl list_queues
Listing queues ...
celery 1
your_product 1
...done.
You can run whole canvas in custom queue:
process_task.apply_async(queue='your_queue')
Try to specify queue_name inside #task decorator. This should help.
Links:
http://docs.celeryproject.org/en/latest/reference/celery.app.task.html
http://docs.celeryproject.org/en/latest/_modules/celery/app/task.html#Task.apply_async