I want to show my admin user a hyperlink to a running celery task.
I use django_celery_results, but this only contains rows for completed tasks.
I could write a simple view, but I guess there is already a much more professional solution.
Steps:
Admin user selects several rows
My custom admin actino creates an async celery tasks
I want to show the admin user an hyperlink where he can wait for the result.
As soon as the result is available a redirect to the django_celery_results admin view should happen.
Which celery add-on could help?
There is no need for any add-on I think to accomplish this. You need to subscribe your service to get notified of specific events (task-succeeded, task-received and task-failed come first to mind). Look at Monitoring section of Celery documentation. Real-time processing in particular as it gives some example code to start with... This is how Flower works for an example.
Related
I am working on a tool to generate fake data for System Center Operations Manager for internal testing purposes. I wrote a script as part of a discovery that is able to create an instance of any class I want and make SCOM fake-discover it. Currently, I'm using a class for AD Printer. Now the next step is to somehow create alerts on behalf of the Printer. For this, I wrote a rule targeted at the AD Printer, which reads from the logs to detect when it should be fired. The logs are being written to from a PowerShell script. However, I see no results. But when I target the same rule to All Windows Computers, I see the alerts.
From what I understand the rule will run on all agents that have an instance of the target class. Since I fake-discovered the AD Printer on this agent (which also happens to be the Management Server), should the rule not run on this?
Any other suggestions on how I can achieve this are welcome as well.
PS. I probably cannot share any of my code as I am under an NDA, but I can clarify my approach further, if needed.
Yes, the Powershell script should run on the agents which have instances of the AD Printer. I recommend you to check the OperationsManager event log for script errors. The easiest way to generate (fake) alerts is to set up a simple, Event-based text log monitor: one specific word can trigger the unhealthy state (which in turn generates an alert), while another word resets the monitor to the healthy state. You can specify criteria for both events. Look at this blog post for further details.
I'm building an application that is constantly collecting data. I want to provide a customizable alerts system for users where they can specify parameters for the types of information they want to be notified about. On top of that, I'd like the user to be able to specify the frequency of alerts (as they come in, daily digest, weekly digest).
Are there any best practices or guides on this topic?
My instincts tell me queues and workers will be involved, but I'm not exactly sure how.
I'm using Parse.com as my database and will also likely index everything with Lucene-style search. So that opens up the possibility of a user specifying a query string to specify what alerts s/he wants.
If you're using Rails and Heroku and Parse, we've done something similar. We actually created a second Heroku app that did not have a web dyno -- it just has a worker dyno. That one can still access the same Parse.com account and runs all of its tasks in a rake task like they specify here:
https://devcenter.heroku.com/articles/scheduler#defining-tasks
We have a few classes that can handle the heavy lifting:
class EmailWorker
def self.send_daily_emails
# queries Parse for what it needs, loops through, sends emails
end
end
We also have the scheduler.rake in lib/tasks:
require 'parse-ruby-client'
task :send_daily_emails => :environment do
EmailWorker.send_daily_emails
end
Our scheduler panel in Heroku is something like this:
rake send_daily_emails
We set it to run every night. Note that the public-facing Heroku web app doesn't do this work but rather the "scheduler" version. You just need to make sure you push to both every time you update your code. This way it's free, but if you ever wanted to combine them it's simple as they're the same code base.
You can also test it by running heroku run rake send_daily_emails from your dev machine.
I'm developing a website (using a LAMP stack) which must handle many user-made scheduling tasks. It works as following: an user creates an event and sets a date, and others users (as many as 63) may join. A few hours before the set date, the system must email each user subscribed to that event. And that's it.
However, I have never handled scheduling, and the only tools I know (poorly) are cron and at. My plan is to create an at job for each event, which will call a script that gets all subscribers emails and mails them.
My question is: is my plan/design good? Is it scalable? Are there better options that I should be aware of?
Why a separate cron job for each event? I've done something similar thing for a newsletter with a cron job just running once per hour and if there are any newsletters to be sent it just handles them. In your case you'd have a script that runs once every hour and gets a list of users for events that happen in the desired time interval since.
It will work. As far as scalability, at the minimum make sure that the script runs in it's own process so it doesn't bog down the server unnecessarily.
Create a php-cli script perhaps?
I'm doing most of my work in Rails nowadays, and there's a wealth of background processing libraries one of them is Resque it uses the redis server to keep track of the jobs
I found a PHP clone https://github.com/chrisboulton/php-resque
Might be overkill for your use case, but give it a shot perhaps
If you would consider a proper framework that uses an application server (and not a simple webserver), Spring has a task scheduling layer that's simple to use. Scheduling jobs on the server really requires more than what a simple LAMP install can do, but I haven't used PHP in a while so maybe there's an equivalent.
Here's an article that compares some of your options.
I'm trying to understand how and when tasks are cleaned up in celery. From looking at the task docs I see that:
Old results will be cleaned automatically, based on the
CELERY_TASK_RESULT_EXPIRES setting. By default this is set to expire
after 1 day: if you have a very busy cluster you should lower this
value.
But this quote is from the RabbitMQ Result Backend section and I do not see any similar text in the Database Backend section. So my question is: is there a backend agnostic approach I can take for old task clean-up with celery and if not is there a DB Backend specific approach I should take? Incase it makes any difference I'm using django-celery. Thanks.
If you click on the link to the setting doc for CELERY_TASK_RESULT_EXPIRES:
http://docs.celeryproject.org/en/latest/userguide/configuration.html#result-expires
It does say that database supports this, but then you need to run celery beat (there's a default periodic task, called every day, to remove expired results).
The backend docs in the task should probably mention this as well, maybe there should be a dedicated guide for backends too. If you want to lobby for this, then please open up an issue at https://github.com/celery/celery/issues
I'm looking forward to develop an automated notification and logging-off system that
notifies and logs off accounts from a computer. So far I planned an example when a class is
scheduled, except accounts that are registered on the scheduled class. It may
notify the logged-in users a certain period of time before the class time and
log them off just before the class time. Or, it could limit their access, for
example to the printer once the class has started.
So my Question is can I use PowerShell to develop this project ? How far can it be useful, or I should think about using python!
Thanks Fellas!
I'm not sure PowerShell brings anything special to the party. What you are talking about would require a PowerShell session running in the background and perhaps even tying into some sort of eventing, perhaps with the timer class. It might be just as easy to automate something using the task scheduler. At the appointed time check the logged on user and if they don't meet the requirement log them off. You could use PowerShell to create the tasks and handle the processing or any other language really.