I am trying to get recent history of a specific task given a task name, but I'm not sure how to do this. All the documentation I have found mention using AsyncResult and passing it an id. I want to the status/history of all the tasks that have a task name.
Also, when I look into the database all task names are null, but when checking the task name in the django python shell and celery worker logs, the task is named as expected.
Any insights or help would be appreciated.
Related
I'm developing the service for customer's orders monitoring, using MongoDB as the standalone backend for tracking and storage of celery tasks' state. So far, it works well, refreshing and displaying the state of all the tasks submitted by a current customer, e.g. STARTED, SUCCESS, FAILURE.
The monitoring UI could be some format as following.
+----------+--------------+-------------+--------------------+
|task_id |created_at |status |operation |
|----------|--------------|-------------|--------------------|
|[uuid] |[timestamp] |[STARTED] |[DOWNLOAD] [DELETE] |
|[uuid] |[timestamp] |[SUCCESS] |[DOWNLOAD] [DELETE] |
|[uuid] |[timestamp] |[RECEIVED] |[DOWNLOAD] [DELETE] |
|[uuid] |[timestamp] |[FAILURE] |[DOWNLOAD] [DELETE] |
|... |... |... |... |
+----------+--------------+-------------+--------------------+
Now I want to implement this [DELETE] utility, which means the customer could revoke a task being executed, via a HTTP request. Considering state of a task could happed to switch into a SUCCESSor FAILURE or other state if there's a latency of request due to HTTP overhead, is it proper to use app.control.revoke(task_id, terminate=True) ?
UPDATED:
Now I configure worker_state_db='/var/run/celery/worker.state.db' in config file of celery for persistent revokes, and stick to app.control.revoke(..., terminate=True). Is it a right option ? I did realize how this revoke command works when I found related answers here.
Which is the best way to programatically terminate (cancel) a celery task
Revoke a task from celery
Celery Task Custom tracking method
Because the service couldn't know target task state when revoke command being broadcasted, it could be as following.
scenario1: target task state is SUCCESS or FAILURE:
worker node is executing another new task (just say task aaa) when being revoked, and will restart executing task aaa. So I have to synchronize REVOKED status of target task into MongoDB without usage of task_revoked signal
scenario2: target task state is RECEIVED or STARTED or other :
worker node is executing this target task, and task_revoked signal would be triggered. But I failed to use this task_revoked signal to synchronize task status into MongoDB. Thus I tried to manually update MongoDB in the same way of scenario1 given a reply received from app.control.revoke(..., terminate=True, reply=True). But I still got a problem as following.
[# ERROR/MainProcess] Task handler raised error: Terminated(15)
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/billiard/pool.py", line 1774, in _set_terminated
raise Terminated(-(signum or 0))
billiard.exceptions.Terminated: 15
How to solve this problem ? And please correct me if I still got something wrong.
I have it implemented the way you describe. There are a couple of things to note:
If you send the revoke command and the task is already done (either in SUCCESS or FAILURE status), the revoke command won't do anything and simply reply that it doesn't know the task. (And more importantly it also won't terminate the child worker process.) See the code for yourself.
You have to be cautious when using terminate=True, though. Depending on how you configure Celery, the worker that's executing your task might have already started another task when the revoke command is sent. There's an explicit warning in the documentation.
On the other hand, once the Celery worker starts to process your task and you really need to cancel the processing, revoking and killing the worker process is the only option.
Hello dear community !
I am trying to use Rundeck to monitor and administrate my talend Jobs. I don't really get how to make this work...
I bluid the job in Talend which created a test_run.bat, a test_0.1.jar and 3 folders ( src, project_name, items).
I created a project and a Job in Rundeck, and in the job i inserted the directory to the test_run.bat file in the workflow section.
Of course it failed when I executed the job, I'm stuck...
I'd really appreciate if anyone who already worked with rundeck to execute Talend jobs could help me out !
It is possible. However there are more questions to ask before an answer can be given.
What is the full path to the bat or sh file of the job, on the server where it will be run?
Will the job be run on Windows, Linux, or another platform?
What is the script (rundeck) used to execute the job?
Is the same version of java installed there ?
What error did you get ?
Obvious things to check:
Path and permissions
I am facing problem with moodle configurations. I have 2 courses setup and the activity completion set for these courses are 1. manual self comletion and 2. Manual completion by manager.
I have also added required blocks for the same.
I completed the course successfully, marked completed by student as well as manager. In the status block it shows block image
Need help to get the status showing complete. I dont know what exactly i am missing.
You should start the Moodle cron tasks in order to update the completion status.
You can manually start the tasks either by running (only if you are an admin) a command from your browser:
http://your.site/admin/cron.php
or by running a command line in you system (from a terminal), like:
/path/to/your/moodle/installation/admin/cli/cron.php
A list of predefined cron tasks of you system can be found under:
http://your.site/admin/tool/task/scheduledtasks.php
or going under Site administration->Server->Scheduled tasks
I have created a sitecore command which triggers an index rebuild.
I would like to be able to inject services with autofac.
Therefore I have followed this tutorial : http://maze-dev.blogspot.be/2014/03/dependency-injection-in-custom-sitecore.html
After having everything in place, it seems like the sitecore scheduling task tries to create a new instance of this command. While these already injected in the commandconfiguration class.
Is there anything else that needs to be done?
The problem is that a Sitecore scheduled task runs in a separate thread, and since the command is registered as InstancePerLifetimeScope (if following the example in the linked blog post), Autofac will inject a new instance in the scheduled task.
Instead, in your scheduled task you should probably get the command from the CommandManager, using something like:
var command = CommandManager.GetCommand("mynamespance:mycategory:mycommand");
and then call Execute on the command.
Now, since the CommandConfigurator at bootstrap time registers the resolved command instance in the static CommandManager, the instance can effectively be seen as a singleton, and it should be available fully injected in the scheduled task (if the command is retrieved through the CommandManager, that is.) If the command is also executed from elsewhere in your Sitecore solution, it will most likely be on another thread. In that case it is probably a good idea to consider if your command implementation is thread safe.
I have a Team City build configuration that calls a powershell script via command line that executes a bunch of database scripts.
What I want to do is to pass a parameter to the command line that is the date of the last SUCCESSFUL build run for the configuration, so I can only execute the db scripts that have been modified since the last run.
I can't seem to find any way to do this. Has anyone ever done this?
You could use the TeamCity REST API to get a list of last successful execution for your specific build. Then knowing the last successful build you can pull the date from the XML.
See here for API Details: http://confluence.jetbrains.com/display/TW/REST+API+Plugin#RESTAPIPlugin-BuildLocator