I'm trying to implement a basic but flexible "event queue" so that my web app can go run subs asynchronously. The way I have working for me now is the app writes a record to an "event queue" table in my database with info on what to do and when to fire. Then I have a daemon script that queries that table periodically and if something needs to be done, it fires it off.
I'd like to start moving towards watchers using something like AnyEvent or EV so I'm not hitting my database so often, but try as I might, I can't find good info on "best practices" for setting something like this up. AnyEvent's documentation is pretty good, but it seems to assume you know how your events should be passed around...which I don't.
What should my watcher be watching? A file? If so, what should be in that file? I don't need to send a whole bunch of data around, I just basically need something that says, "go off and run this sub right now"
I would greatly appreciate someone pointing me in the right direction.
EDIT:
It's been requested that I be more specific: The events I'm trying to fire are various. Sometimes it's an email that needs sending, sometimes it's some DB work, sometimes I just need an action to be delayed for a few hours or days. In all cases, I have some sort of backend script to handle the action, I just need a way for my frontend (web app) to tell my backend, "hey I need you to go do this in x minutes" or "I need you to do this now"
You might look at Mojolicious which is built to be non-blocking and respond asynchronously. It even uses EV internally if you have it installed. To get started read the doc for Mojolicious::Lite and then the Guides (in order) from here: http://mojolicio.us/perldoc
So what is it that you are actually trying to do? Until you describe what it is you actually need, I can't help you any further that that.
There are some CPAN modules in order to run asynchronous tasks via message queues. Examples:
Queue::DBI
POE::Component::MessageQueue
Any::MQ
Related
In my conversation dialogflow, I would like to add some progress messages like hang in with me, I'm looking up for that data or similar in the conversation. Is there any guidance or best practice to do this?
Unfortunately, there is no good way to do this at this time. If your webhook takes longer than about 5 seconds, Dialogflow will return one of the default responses it is set with. If you're not using Dialogflow, the Action SDK will say your webhook isn't responding and will close the conversation.
There is currently no way to send a reply, and then send another reply without the user saying something first.
One workaround might be to have the default response be something like "I'm looking that information up. Ask me again in a few seconds." When your lookup finally completes, cache the information so when/if the user asks the question again, you can return it more quickly.
Depending how long it takes, you may also wish to register a dynamic reprompt. This will send an event to your webhook if the user doesn't say anything. In a situation like this, they may say nothing for a few seconds, but that may be long enough for you to have computed the reply. So after a few seconds of silence you can suddenly announce "I've figured it out, the answer you were looking for is..." or something similar. This has some limitations - you can only reprompt twice like this before Google sends you a final reprompt and closes the conversation.
Although the platform does support notifications, these are still in developer preview and don't work with all devices. They also don't quite continue the conversation (it doesn't just start talking) - they just send a notification to a phone that there is a message and that they can restart the conversation. Depending on your use case, this may be useful combined with the above.
Update
The Media Response includes a feature that we can take advantage of to handle this. Similar to the dynamic reprompt method above, you'll get a call automatically when the media you're playing ends. So you can play a short "hold music" and your webhook will be called when it is finished. You can then either give the result or say you're still working on it and play more hold music.
I have an application (Laravel + MongoDB running on Nginx) where I pull some data from the database and render it on the screen. The application focusses on multiple real life objects. Once an object is turned on (is_on equals to true in the database), the timer on the screen needs to start ticking. Once the object is turned off (is_on equals to false in the database) the clock stops ticking and resets to 0. The format of the clock is HH:MM:SS. So it shows how long the real life object is turned on.
My problem is that I don't really now how to save/implement such timer. When the user request the page, I pull the necessary data from the database. If I also save the timer in the database, you have to make a query every second which is very bad practice.
I remembered something about WebSockets and tried to look into them. I actually managed to build a basic Hello World chat application, but don't really know how to implement this in my project. There is no place for it in the database (because of the queries), so I don't really know where to save that timer on the server. I'm also doubting if WebSockets are the way to go.
So are WebSockets the way to go and if it is, can you guys point me in the right direction on how to implement this? If not, can you advise me what I should do?
Thanks in advance!
From your question:
I understand that the objects you print in the screen are modified by
users in the application, and your aim is to live forward those
modifications to other active client instances of your application.
In that case, as you mention, I would point you to websockets. They are a great way to feed information directly to the client, so the client receives the update signals and modify the interface, with no need of user action.
In order to implement the logic to notify the client, I recommend using a push approach, but this is really depending on what kind of clients you'd like to notify, since the push world is still a bit tricky.
Further readings for this websocket based push implementation:
Question about Push Flags:
Difference between push and urgent flags in TCP
If your client runs in browser or mobile this question is nice to read:
How to send push notification to web browser?
Also html5 websockets:
http://www.websocket.org/aboutwebsocket.html
As a sidenote:
A nice architecture for client-server live communication is based on node.js together with socket.io library offering good performance and not really complex implementation if you know what you do.
I am going to use Laravel 4 queues and integrate them with Iron.io
All of that is pretty straight forward, and I dont think I am going to have problems with that.
Thing that interests me is what is the best way to get status once task is completed?
Iron.io is going to do return call to my server to trigger job, and once that job completes I need to notify user about that...
How could I store this responses, and still be aware of job its related to, because there will be number of different job types?
I would like to hear how did you implement this.
Thanks
As Joseph pointed out in the comments, you'll need to:
Hopefully have a job quick enough for it to be finished when a user is still in browser
Use some sort of way to PUSH data back to your web browser.
The popular methods are:
Websockets
Server-sent events
Long polling - with ajax
I have a perl app which processes text files from the local filesystem (think about it as an overly-complicated grep).
I want to design a webapp which allows remote users to invoke the perl app by setting the required parameters.
Once it's running it would be desirable some sort of communication between the perl app and the webapp about the status of the process (running, % done, finished).
Which would be a recommended way of communication between the two processes? I was thinking in a database table, but I'm not really sure it's a good idea.
any suggestions are appreciated.
Stackers, go ahead and edit this answer to add code examples or links to them.
DrNoone, two approaches come to mind.
callback
Your greppy app needs to offer a callback function that returns the status and which is periodically called by the Web app.
event
This makes sense if you are already using a Web server/app framework which exposes an event loop usable from external applications (rather unlikely in Perl land). The greppy app fires events on status changes and the Web app attaches/listens to them and acts accordingly.
For IPC as you envision it, a plain database is not so suitable. Look into message queues instead. For great interop, pick AMPQ compliant implementation.
If you run the process using open($handle, "cmd |") you can read the results in real time and print them straight to STDOUT while your response is open. That's probably the simplest approach.
We have a predominantly RESTful architecture for our product. This enables us to nicely implement almost all of the required functionality, except this new requirement thats come in.
I need to implement a page which lets the user to large scale DB operations synchronously. They can stop the operation in between, if they realized they made a mistake (rather than waiting for it to complete and doing an undo operation)
I was wondering if some one could give some pointers as to what would be the best way to implement such a functionality?
Cheers!
Nirav
How about a resource that encapsulates a set of batch operations? Creating the resource means kicking off the operations (data to indicate what the operations should do is submitted via POST). Updating the resource allows stopping it or modifying it while processing.
I would kick off the large operation in a separate thread. Show the user a constantly updated status of the thread, along with a Cancel button. If the user clicks the Cancel button, you kill thread.
This is the way I've implemented similar things in the past.
The idea is to give them their control back immediately, but don't let them do anything else until the thread is complete except cancel.
In generic terms, you need a "job queue" and a way to manage the queue.
You need to integrate a batch manager, or implement your own. There are several products that can help you. As an example, read this article
http://www.ibm.com/developerworks/websphere/techjournal/0801_vignola/0801_vignola.html