I've searched extensively, but cannot find anything on this.
Is it possible to set a celery task priority for a celery beat task, when using add_periodic_task() something like:
sender.add_periodic_task(
crontab(minute="*/5"),
self_monitor_health_checks,
priority=3,
)
Thanks for your help and input.
It turns out that the above does work, despite it being undocumented. You can set a priority like this:
sender.add_periodic_task(
crontab(minute="*/5"),
self_monitor_health_checks,
priority=3,
)
Related
I use Akka Stream on Scala. I'd like to set a scheduler which runs on every 24:00. I tried to search for it. But I could't find what I want to do. Could you tell me how to write code?
This is mentioned in a comment but should really be the preferred solution using only akka-streams:
Source.tick(0.seconds, 24.hours, Done).runForeach { x =>
//do something
}
Use the build in Akka scheduler, see:
http://doc.akka.io/docs/akka/current/scala/scheduler.html
You can use the scheduler like:
system.scheduler.schedule(
initialDelay = FiniteDuration(/*offset to next 24:00*/),
interval = FiniteDuration(24, TimeUnit.HOURS),
receiver = self,
message = ScheduleAkkaStream
)
Then in the actor, when the ScheduleAkkaStream is received, run the job
The most commonly used one is akka quartz scheduler:
https://github.com/enragedginger/akka-quartz-scheduler
This one written by me and has no additional dependencies, a bit more lightweight than using quartz with fewer bells and whistles:
https://github.com/johanandren/akron
I used:
system.scheduler.scheduleWithFixedDelay(10.seconds, 30.seconds)(
() => {
println("Action")
}
)
Specifically I use shared_task decorator to create a celery task.
I tried the obvious #shared_task(expires=3) but it doesn't seem to work.
Is there a way to tell this task is supposed to expire ** seconds after it is received as you can do with apply_async in calling time?
I am not sure about shared_task, but in Celery 5.2.7 at least, you can pass an expires parameter to #app.task.
from myapp.celery import app as celery_app
#celery_app.task(expires=60)
def my_task(...
Probably this should help
from tasks import add
result = add.apply_async(args=[10, 10], expires=6000)
Also make sure that your producer and consumer machine clocks are in sync. Else use
CELERY_ENABLE_UTC = True
CELERY_TIMEZONE = 'Etc/UTC'
I have a django project in combination with celery and my need is to be able to schedule tasks dynamically, at some point in the future, with recurrence or not. I need the ability to delete/edit already scheduled tasks
So to achieve this at the beginning I started using django-celery with DatabaseScheduler to store some PeriodicTasks (with expiration) to the database as it is described more or less here
In this way if I close my app and start it again my schedules are still there
My problem though still remains since I cannot utilize the eta and schedule a task at some point in the future. Is it possible somehow to dynamically schedule a task with eta?
A second question of mine is whether I can schedule a once off task, like schedule it to run e.g. at 2015-05-15 15:50:00 (that is why I'm trying to use eta)
Finally, I will be scheduling some thousants of notifications, is celery beat capable to handle this number of scheduled tasks? some of them once-off while others being periodic? Or do I have to go with a more advanced solution such as APScheduler
Thank you
I've faced the same problem yesterday. My ugly temporary solution is:
# tasks.py
from djcelery.models import PeriodicTask, IntervalSchedule
from datetime import timedelta, datetime
from django.utils.timezone import now
...
#app.task
def schedule_periodic_task(task='app.tasks.task', task_args=[], task_kwargs={},
interval=(1, 'minute'), expires=now()+timedelta(days=365*100)):
PeriodicTask.objects.filter(name=task+str(task_args)+str(task_kwargs)).delete()
task = PeriodicTask.objects.create(
name=task+str(task_args)+str(task_kwargs), task=task,
args=str(task_args),
kwargs=str(task_kwargs),
interval=IntervalSchedule.objects.get_or_create(
every=interval[0],
period=interval[1])[0],
expires=expires,
)
task.save()
So, if you want to schedule periodic task with eta, you shoud
# anywhere.py
schedule_periodic_task.apply_async(
kwargs={'task': 'grabber.tasks.grab_events',
'task_args': [instance.xbet_id], 'task_kwargs': {},
'interval': (10, 'seconds'),
'expires': instance.start + timedelta(hours=3)},
eta=instance.start,
)
schedule task with eta, which creates periodic task. Ugly:
deal with raw.task.name
strange period (n, 'interval')
Please, let me know, if you designed some pretty solution.
I am an RTOS newbie and I am creating a simple real time system for automotive
I am wondering if it possible to create a task inside another task.
I tried to do this by the following method but it doesn't work.
void vTask1 { *pvParameters){
unsigned portBASE_TYPE taskPriority;
taskPriority=uxTaskPriorityGet( NULL );
char x;
while (1){
x= 5 ;
if (x==5)
xTaskCreate( vTask2 , "task2", 1000, "task2 is running", taskPriority+5 , NULL );
}
when I debug that code it hangs at xTaskCreate without executing the new task
and I searched the manual and the internet for something about this but I didn't find any.
would anyone tell me is that possible to do in RTOS or I am doing it in a wrong way?
Tasks can be created before the scheduler has been started (from main), or after the scheduler has been started (from another task). The xTaskCreate() API documentation is here:
http://www.freertos.org/a00125.html . You will also find a set of demo tasks that demonstrate creating and deleting tasks from another task in the main FreeRTOS .zip file download. Look in the FreeRTOS/Demo/Common/Minimal/death.c file (death for suicidal tasks as they delete themselves after creation).
If xTaskCreate() returns NULL then you will probably have run out of heap space. See http://www.freertos.org/a00111.html. I think most of the hundreds or pre-configured examples that come in the zip file download have comments to this effect.
Check the return value of xTaskCreate api.
one more thing the second task which you are creating is vtask2 which is having lower priority than vtask1 the one who is creating . And vtask1 is running in while(1) scheduler will not schedule vtask2.
you can delay or suspend the vtask1 after creating vtask2.
then vtask2 may execute.
I have a task in Celery that could potentially run for 10,000 seconds while operating normally. However all the rest of my tasks should be done in less than one second. How can I set a time limit for the intentionally long running task without changing the time limit on the short running tasks?
You can set task time limits (hard and/or soft) either while defining a task or while calling.
from celery.exceptions import SoftTimeLimitExceeded
#celery.task(time_limit=20)
def mytask():
try:
return do_work()
except SoftTimeLimitExceeded:
cleanup_in_a_hurry()
or
mytask.apply_async(args=[], kwargs={}, time_limit=30, soft_time_limit=10)
This is an example with decorator for an specific Task and Celery 3.1.23 using soft_time_limit=10000
#task(bind=True, default_retry_delay=30, max_retries=3, soft_time_limit=10000)
def process_task(self, task_instance):
"""Task processing."""
pass