I just want to get some sort of notification in my perl script when particular file is modified.
Is there any other way rather than just polling it constantly?
Win32::ChangeNotify, Linux::Inotify2
AnyEvent::Filesys::Notify: Check out the section which says "Why Another Module For File System Notifications".
Excerpt:
AnyEvent::Filesys::Notify exists because I need a way to simply tie
the functionality those modules provide into an event framework.
Neither of the existing modules seem to work with well with an event
loop. Filesys::Notify::Simple does not supply a non-blocking interface
and File::ChangeNotify requires you to poll an method for new events.
Related
Currently, I'm using the powerful SetWinEventHook() function to catch some user-interface's actions like minimizing and maximizing from other window runned by programs on the computer.
So I inspired myself by using the code provided by BrendanMcK on this post and it does work (I mean: the callback function is called when an event occurs) until the line
MessageBox.Show("Something")
is present. But I don't want to use any form or window for this program..
After some research, I figured out this kind of hook needs a message loop to allow the redirection of messages from other window handles. Apparently, calling the thread using Application.Run() should do the trick, but I would prefer something cleaner, in the C# object itself.
So my question is: is it possible to create a message loop inside an object's method?
http://bytes.com/topic/c-sharp/answers/557342-thread-message-loop-c
No, the function doesn't require a window handle so no "form" is needed. But the MSDN docs for the function is quite explicit:
The client thread that calls SetWinEventHook must have a message loop in order to receive events.
A message loop is universal in any program that want to receive notifications that are generated externally by other processes or threads. It is the common solution to the producer-consumer problem. Clearly any GUI app has a need for such a solution, Windows messages are generated by the operating system. It isn't different for SetWinEventHook(), the accessibility events originate in other programs. There is no clean mechanism to "interrupt" a thread and make it run other code, the re-entrancy problems that causes are extremely difficult to deal with. The thread has to co-operate, it must be idle and ready to receive a notification to safely process it. A message loop solves that problem.
Pumping a message loop (calling Application.Run) in a method is certainly possible. But do keep in mind that the method won't return until you explicitly stop the loop with Application.ExitThread. There is therefore usually only one good place for that call, the Main() method of your program.
Starting your project with a Winforms or WPF project template is a very good way to get this right. You have no need to actually create a window, call Application.Run() without an argument, after pinvoking SetWinEventHook.
I'm trying to implement a basic but flexible "event queue" so that my web app can go run subs asynchronously. The way I have working for me now is the app writes a record to an "event queue" table in my database with info on what to do and when to fire. Then I have a daemon script that queries that table periodically and if something needs to be done, it fires it off.
I'd like to start moving towards watchers using something like AnyEvent or EV so I'm not hitting my database so often, but try as I might, I can't find good info on "best practices" for setting something like this up. AnyEvent's documentation is pretty good, but it seems to assume you know how your events should be passed around...which I don't.
What should my watcher be watching? A file? If so, what should be in that file? I don't need to send a whole bunch of data around, I just basically need something that says, "go off and run this sub right now"
I would greatly appreciate someone pointing me in the right direction.
EDIT:
It's been requested that I be more specific: The events I'm trying to fire are various. Sometimes it's an email that needs sending, sometimes it's some DB work, sometimes I just need an action to be delayed for a few hours or days. In all cases, I have some sort of backend script to handle the action, I just need a way for my frontend (web app) to tell my backend, "hey I need you to go do this in x minutes" or "I need you to do this now"
You might look at Mojolicious which is built to be non-blocking and respond asynchronously. It even uses EV internally if you have it installed. To get started read the doc for Mojolicious::Lite and then the Guides (in order) from here: http://mojolicio.us/perldoc
So what is it that you are actually trying to do? Until you describe what it is you actually need, I can't help you any further that that.
There are some CPAN modules in order to run asynchronous tasks via message queues. Examples:
Queue::DBI
POE::Component::MessageQueue
Any::MQ
I am trying to find a way to monitor the contents of a directory for changes. I have tried two approaches.
Use kqueue to monitor the directory
Use GCD to monitor the directory
The problem I am encountering is that I can't find a way to detect which file has changed. I am attempting to monitor a directory with potentially thousands of files in it and I do not want to call stat on every one of them to find out which ones changed. I also do not want to set up a separate dispatch source for every file in that directory. Is this currently possible?
Note: I have documented my experiments monitoring files with kqueue and GCD
My advice is to just bite the bullet and do a directory scan in another thread, even if you're talking about thousands of files. But if you insist, here's the answer:
There's no way to do this without rolling up your sleeves and going kernel-diving.
Your first option is to use the FSEvents framework, which sends out notifications when a file is created, edited or deleted (as well as things to do with attributes). Overview is here, and someone wrote an Objective C wrapper around the API, although I haven't tried it. But the overview doesn't mention the part about events surrounding file changes, just directories (like with kqueue). I ended up using the code from here along with the header file here to compile my own logger which I could use to get events at the individual file level. You'd have to write some code in your app to run the logger in the background and monitor it.
Alternatively, take a look at the "fs_usage" command, which constantly monitors all filesystem activity (and I do mean all). This comes with Darwin already, so you don't have to compile it yourself. You can use kqueue to listen for directory changes, while at the same time monitoring the output from "fs_usage". If you get a notification from kqueue that a directory has changed, you can look at the output from fs_usage, see which files were written to, and check the filenames against the directory that was modified. fs_usage is a firehose, so be prepared to use some options along with Grep to tame it.
To make things more fun, both your FSEvents logger and fs_usage require root access, so you'll have to get authorization from the user before you can use them in your OS X app (check out the Authorization Services Programming Guide for info on how to do it).
If this all sounds horribly complicated, that's because it is. But at least you didn't have to find out the hard way!
I have a perl app which processes text files from the local filesystem (think about it as an overly-complicated grep).
I want to design a webapp which allows remote users to invoke the perl app by setting the required parameters.
Once it's running it would be desirable some sort of communication between the perl app and the webapp about the status of the process (running, % done, finished).
Which would be a recommended way of communication between the two processes? I was thinking in a database table, but I'm not really sure it's a good idea.
any suggestions are appreciated.
Stackers, go ahead and edit this answer to add code examples or links to them.
DrNoone, two approaches come to mind.
callback
Your greppy app needs to offer a callback function that returns the status and which is periodically called by the Web app.
event
This makes sense if you are already using a Web server/app framework which exposes an event loop usable from external applications (rather unlikely in Perl land). The greppy app fires events on status changes and the Web app attaches/listens to them and acts accordingly.
For IPC as you envision it, a plain database is not so suitable. Look into message queues instead. For great interop, pick AMPQ compliant implementation.
If you run the process using open($handle, "cmd |") you can read the results in real time and print them straight to STDOUT while your response is open. That's probably the simplest approach.
I am using the wonderful AnyEvent for creating an asynchronous TCP server (specifically, a MUD server).
In order to keep everything running smoothly and with as few blocking/synchronous pieces of code possible, I have replaced some modules I was using with their asynchronous counterpart, for example AnyEvent::Memcached and AnyEvent::Gearman. This allows the main program to be quite speedy, which is desirable. I have coded around the need for some of these calls to be synchronous.
One problem I currently have, and the focus of this question, is logging.
Before turning to AnyEvent for this server program, I was using Log::Log4perl as it allows me to fine-tune which modules or subroutines should be logged, at which level and to which log output (screen, file, etc).
The problem here is that the Log4perl actions (warn, info, etc) are currently performed synchronously but I have no requirement for that as long as the log lines eventually end up on the screen / file (and in the correct order).
Is Log::Log4perl still the right choice when using an asynchronous event handler such as AnyEvent, or should I look at a different module? If so, which is recommended?
AnyEvent::Log, which comes with AnyEvent, uses AnyEvent::IO, which appends to files asynchronously when IO::AIO is available (and synchronously when not).
What you are trying to avoid? If it's synchronous file IO (writing to log files/stdout etc.) then your problem would probably be solved with an asynchronous and/or buffering appender(s) rather than replacing all use of Log4perl in your code.
Log::Log4perl::Appender::Buffer seems like it might be a good start, but a completely async appender doesn't appear to exist anymore.