I have this Perl software that is supposed to run 24/7. It keeps open a connection to an IMAP server, checks for new mail and then classifies new messages.
Now I have a user that is hibernating his XP laptop every once in a while. When this happens, the connection to the server fails and an exception is triggered. The calling code usually catches that exception and tries to reconnect. But in this case, it seems that Windows (or Perl?) is catching the exception and delivering it to the user via a message box.
Anyone know how I can prevent that kind of wtf? Could my code catch a "system-is-about-to-hibernate" signal?
To clear up some points you already raised:
I have no problem with users hibernating their machines. I just need to find a way to deal with that.
The Perl module in question does throw an exception. It does something like "die 'foo bar'. Although the application is completely browser based and doesn't use anything like Wx or Tk, the user gets a message box titled "poll_timer". The content of that message box is exactly the contents of $# ('foo bar' in this example).
The application is compiled into an executable using perlapp. The documentation doesn't mention anything about exception handling, though.
I think that you're dealing with an OS-level exception, not something thrown from Perl. The relevant Perl module is making a call to something in a DLL (I presume), and the exception is getting thrown. Your best bet would be to boil this down to a simple, replicable test case that triggers the exception (you might have to do a lot of hibernating and waking the machines involved for this process). Then, send this information to the module developer and ask them if they can come up with a means of catching this exception in a way that is more useful for you.
If the module developer can't or won't help, then you'll probably wind up needing to use the Perl debugger to debug into the module's code and see exactly what is going on, and see if there is a way you can change the module yourself to catch and deal with the exception.
It's difficult to offer intelligent suggestions without seeing relevant bits of code. If you're getting a dialog box with an exception message the program is most likely using either the Tk or wxPerl GUI library, which may complicate things a bit. With that said, my guess would be that it would be pretty easy to modify the exception handling in the program by wrapping the failure point in an eval block and testing $# after the call. If $# contains an error message indicating connection failure, then re-establish the connection and go on your way.
Your user is not the exception but rather the rule. My laptop is hibernated between work and home. At work, it is on on DHCP network; at home, it is on another altogether. Most programs continue to work despite a confusing multiplicity of IP addresses (VMWare, VPN, plain old connection via NAT router). Those that don't (AT&T Net Client, for the VPN - unused in the office, necessary at home or on the road) recognize the disconnect at hibernate time (AT&T Net Client holds up the StandBy/Hibernate process until it has disconnected), and I re-establish the connection if appropriate when the machine wakes up. At airports, I use the local WiFi (more DHCP) but turn of the wireless altogether (one physical switch) before boarding the plane.
So, you need to find out how to learn that the machine is going into StandBy or Hibernation mode for your software to be usable. What I don't have, I'm sorry to say, is a recipe for what you need to do.
Some work with Google suggests that ACPI (Advanced Configuration and Power Interface) is part of the solution (Microsoft). APM (Advanced Power Management) may also be relevant.
I've found a hack to avoid modal system dialog boxes for hard errors (e.g. "encountered and exception and needs to close"). I don't know if the same trick will work for this kind of error you're describing, but you could give it a try.
See: Avoiding the “encountered a problem and needs to close” dialog on Windows
In short, set the
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Windows\ErrorMode
registry key to the value “2″.
Related
I have to design windows service and I have some question:
Proper error handling, if there is an error what happens to the
service? Does he continue to be up and logging in? Error recording
in event viewer? Is he falling?
What happens to a long run? How do you know for sure that everything
is running as required, and he is not stuck?
How to handle high memory consumption, out of memory, or other error
that wasn't been write to the log?
Handle users - what happened if create log to user A and changed to
user B? Rewrite or continue from same point?
How to handle times? - Is the service automatically up?
Thank you.
For error handling, the best I can recommend is taking advantage of try / catch cases. This way you ensure that you handle the cases where something unexpected happens and you can either try to correct it or bring the service down cleanly. Keep in mind that exceptions are not propagated outside the scope of a thread so you need to handle them in each thread.
To be able to tell if the service is doing fine, you can periodically log into the Event Log what the service does. If you do proper try / catch for each thread, it should go smoothly. In C# you can use log4net and the EventLogAppender to log crucial / error info in the Event Log.
If your service causes high memory usage for no apparent reason, it is likely a memory leak. Microsoft has a free tool called ".Net CLR profiler" that allows you to properly profile your service and see what exactly is causing the leak.
Unless you are dealing with user-protected files (in which case you need to consult the Log On tab of your service to give it the appropriate credentials), your service shouldn't depend on any logged-in user. Services run independently of the users on the computer.
A service can be set to start automatically, to start only on-demand, or to simply be disabled completely.
I'm establishing a watch via the socket interface, and then subscribing to changes.
For each incoming PDU, if the map has a "warning" key, I output the warning to the console/user, as the docs suggest.
However, when an overflow happens, it looks like I don't get the "warning" key just once, but instead every incoming PDU has the same warning ("recrawl happened 1 time") over and over (AFAICT?), so I end up spamming the console with the same error message.
For me, it'd be preferable if Watchman only sent the "warning" key once per overflow event. Otherwise I'm looking at having to cache the "warnings already shown to the user" to avoid spamming the console.
Also, in terms of overflow behavior in general, the warning says:
To resolve, please review the information on
https://facebook.github.io/watchman/docs/troubleshooting.html#recrawl
To clear this warning, run:
`watchman watch-del ... ; watchman watch-project ...`
But I'd prefer to have a way to reset the warning without having to cancel and resubscribe my subscription. E.g. right now I have to control-c kill my program, run the watchman watch-del command, then restart my program.
Which I could automate internally, e.g. have my program detect the "overflow happened" warning message, kill it's subscription, issue a watch-del, and then re-issue the watch.
But, even if I could reset the warning via the socket interface, or do this internal watch-del, I'm wondering why the warning needs reset at all--e.g. in theory if watchman has already done a recrawl, and I told the user it happened (by logging it to the console), shouldn't things be fine now? Why is the watch-del + re-watch required in the first place?
E.g. as long as overflows are not happening constantly, it seems like watchman doing the recrawl (so gets back in sync with the file system) + issuing one warning PDU should mean everything is back to normal, and my user program could, ideally, stay dumb/simple and just keep getting the post-overflow/post-recrawl PDUs on it's same/existing subscription.
Sorry that this isn't as clear as it could be.
First: you don't strictly have to take action here, as the watchman service already recovered from the overflow.
It's just advising you that you may have a local configuration problem; if you are on Linux you might consider increasing the various inotify sysctl parameters. If you are on the mac there is very little you can do about this. The warning is sticky so that it is forced in the face of the user. We later had users request a way to suppress it, so the suggestion for deleting and restarting the watch was added.
The result was a pretty noisy warning that caused more confusion.
In watchman 4.7 we added a configuration option to turn off this warning: https://facebook.github.io/watchman/docs/config.html#suppress_recrawl_warnings
The intent here is to hide the warning from users that don't know how (or don't have permission) to remediate the system configuration. It works well in conjunction with the (undocumented) perf sampling configuration options that can record and report the volume of recrawls to a site-specific reporting system.
To set the seem, I'm an experienced developer and have coded many languages over the years, including a good bit of Perl back in late 90's early 00's. Since then I haven't touched Perl, but now have a client who wants some changes making to an existing open source project built using Perl5 and Catalyst. I've quickly worked through the Catalyst tutorials, read a few books online and am now starting to feel my way.
I have the existing project up and running on a clean Debian Wheezy VM and am testing the code an my changes using the Catalyst Development Server.
While working through the tutorials and writing a few test apps, the development server would always output a lot of useful information when run, such as the configured routes etc.. But under this project, when I run the server I don't get a lot of output. I don't even get messages sent to $c->log->debug();
I run the server with the following command:
perl ./script/asnn_panel_server.pl -d -r
Which outputs the following:
HTTP::Server::PSGI: Accepting connections at http://0:3000/
I can access the server and the application is running fine.
In a test controller action I can try the following lines:
$c->log->debug("A test debug message");
print "A test print message\n";
The debug log message does not appear in my development server output, but the print line does. So I know the call to $c->log->debug() is not blowing up, because the next line is executing, but where is it going?
So essentially I feel I 'could' get more useful output from the Catalyst Development server, but am not.
I have googles but can't find anything of relevance. Sorry if I'm going in the wrong direction here, I do know what I doing in general, but have a lot to pick up here in a short amount of time!
I suspect my issues might be specific to the open source project I'm working on, but there's not a lot of help to be had from that direction. Could anyone give me any pointers as to what to investigate?
UPDATE : I now realise that the application is using log4perl, which is configured to send $c->log->debug() to syslog. I still don't know why the Catalyst Development server isn't providing much output.
:wq
For anyone coming upon this later, if you want to see the developer debug stream (stuff about the routes and classes and models your application is using, etc you need to be in debug mode, which you can do easily by setting CATALYST_DEBUG=1 in your env (I often start my app like "CATALYST_DEBUG=1 perl -Ilib script/myapp_server.pl"
There is sadly a difference between debug as a log level and debugging mode. The way catalyst works is that if you are in debugging mode (via CATALYST_DEBUG=1, or any of the other documented ways this gets turned on) all this debugging stream gets sent to the log, most of it logged at the debug level (again debug as a log level is distinct from debugging developer mode :( ) So you need both debugging mode and your logger should be set to listen at the debug level.
If you use the default catalyst log, it is debug level by default, so doing CATALYST_DEBUG=1 is all you need. If you use a different logger be sure to enable debug log level for your development setup, if you wish to see those developer stream logs.
Messages sent to $c->log->debug() are generally disabled in production environments. If it doesn't seem to matter whether you start your scripts with or without the -d switch, then I'd suggest something downstream in the sequence is setting the environment variable CATALYST_DEBUG to 0 or undef unilaterally.
That said, you should be able to see the output of $c->log->info() or $c->log->warn() calls. The answer to that question should help you determine if the problem is log4perl or Catalyst related.
Hopefully that will get you on your way.
I have a process, running on Solaris 10, that is terminating due to a SIGSEGV. For various uninteresting reasons it is not possible for me to get a backtrace by the usual means (gdb, backtrace call, corefile are all out). But I think dtrace might be useable.
If so, I'd like to write a dtrace script that will print the thread stacks of the process when the process is killed. I'm not very familiar with dtrace, but this seems like it must be pretty easy for someone who knows it. I'd like to be able to run this in such a way as to monitor a particular process. Any thoughts?
In case anyone else stumbles across this, I'm making some progress experimenting on OS X with the following script I've cobbled together:
#!/usr/sbin/dtrace -s
proc:::fault
/pid == $1/
{
ustack();
}
I'll update this with a complete solution when I have one.
A couple of Solaris engineers wrote a script for using Dtrace to capture crash data and published an article on using it, which can now be found at Oracle Technology Network: Enabling User-Controlled Collection of Application Crash Data With DTrace.
One of the authors also published a number of updates to his blog, which can still be read at https://blogs.oracle.com/gregns/, but since he passed away in 2007, there haven't been any further updates.
I am writing a couple fo scripts that go and collect data from a number of servers, the number will grow and im trynig to future proof my scripts, but im a little stuck.
so to start off with I have a script that looks up an IP in a mysql database and then connects to each server grabs some information and then puts it into the database again.
What i have been thinknig is there is a limited amount of time to do this and if i have 100 servers it will take a little bit of time to go out to each server get the information and then push it to a db. So I have thought about either using forks or threads in perl?
Which would be the prefered option in my situation? And hs anyone got any examples?
Thanks!
Edit: Ok so a bit more inforamtion needed: Im running on Linux, and what I thought was i could get the master script to collect the db information, then send off each sub process / task to connect and gather information then push teh information back to the db.
Which is best depends a lot on your needs; but for what it's worth here's my experience:
Last time I used perl's threads, I found it was actually slower and more problematic for me than forking, because:
Threads copied all data anyway, as a thread would, but did it all upfront
Threads didn't always clean up complex resources on exit; causing a slow memory leak that wasn't acceptable in what was intended to be a server
Several modules didn't handle threads cleanly, including the database module I was using which got seriously confused.
One trap to watch for is the "forks" library, which emulates "threads" but uses real forking. The problem I faced here was many of the behaviours it emulated were exactly what I was trying to get away from. I ended up using a classic old-school "fork" and using sockets to communicate where needed.
Issues with forks (the library, not the fork command):
Still confused the database system
Shared variables still very limited
Overrode the 'fork' command, resulting in unexpected behaviour elsewhere in the software
Forking is more "resource safe" (think database modules and so on) than threading, so you might want to end up on that road.
Depending on your platform of choice, on the other hand, you might want to avoid fork()-ing in Perl. Quote from perlfork(1):
Perl provides a fork() keyword that
corresponds to the Unix system call of
the same name. On most Unix-like
platforms where the fork() system call
is available, Perl's fork() simply
calls it.
On some platforms such as Windows
where the fork() system call is not
available, Perl can be built to
emulate fork() at the interpreter
level. While the emulation is
designed to be as compatible as
possible with the real fork() at the
level of the Perl program, there are
certain important differences that
stem from the fact that all the pseudo
child "processes" created this way
live in the same real process as far
as the operating system is concerned.