I am finding seemingly random ordering of print versus python logging outputs to the pydev console.
So for example here is my code:
import logging
logging.warning('Watch out!') # will print a message to the console
print "foo"
logging.info('I told you so') # will not print anything
logging.warning("Event:")
logging.warning("told ya")
Note the print line around "foo". What is bizzare is each time I run this the order of output with respect to foo changes! That is foo appears one time at the top another time in the middle another time at the end and so forth.
This is unfortunate in a less toy context when I want to know the sequence in which these output events occured in the code (e.g. when tracing/logging etc). Any suggestions as to what might be going on? Latest pydev, python 2.7.6
Thanks!
p.s. As a side-effect (probably) this is making an eclipse tool out there "grep console" behave oddly). But the problem I describe above is independent of that.
Related
Environment: Linux CentOS 7 #HPC
Interface: command interface, no GUI
My PERL scripts have a logic error. It does not go through in a "foreach" loop. I use debugger command below:
perl -d /scripts_location/perlscripts.pl
However, it is run step by step. My scripts have almost thousand lines. Here is my questions:
How to debug my scripts from specific line?
How to figure out the loop cannot be run? And why the loop cannot be run?
Is there any resource to show debugger skills in a whole process?
I searched online. Most of them explain the commands. But few introduce the debug from the very beginning. I mean that first a simple program is given, set breakpoint or other label in the program, check output or trace error, and so on. After viewing websites, I am unable to know how to start debugging using PERL debugger. I used to debugging my program using output at specific lines to check the output is correct or not. However, current logic error cannot be figured out in this way.
Any further suggestion and help would be highly appreciated.
After starting the debugger, you can tell it to continue until it reaches a given line, e.g.
c 124
To figure out why a foreach loop isn't entered, check the loop's list before entering it. You can tell the debugger to evaluate the expression like this:
x #values
if the loop is something like foreach my $value (#values). It will probably tell you
empty array
To discover why the array is emtpy, you can try to "watch" the array
w #values
and then run the script with r. It will stop once any watched value changes.
Use h to view the help.
I have a custom nagios plugin which is written in Perl. For complicated political reasons I am required to hide the source code of this plugin. The only way I found to do this was by using perlc (http://marginalhacks.com/Hacks/perlc).
In the words of author:
"Takes a single perl script, converts the block using a simple encoding with an optionally defined key. The script is decoded at runtime and fed to the perl library, to avoid it getting in the hands of the user."
The problem I am getting is that Nagios shows "No output returned from plugin" when I used the compiled version of the plugin. The raw perl source works just fine.
After debugging for a while I narrowed the problem down to using exit in perl. I.e
This works fine when compiled.
print "OK: Everything is working fine.\n";
This however does not work and results in ""No output returned from plugin"
print "OK: Everything is working fine.\n";
exit 1;
It doesn't matter how I exit (0 1 2 or 3) I still get the same problem.
According to the cross-posted PerlMonks thread, the issue was resolved by enabling autoflushing:
$| = 1;
From perlvar:
HANDLE->autoflush( EXPR )
$OUTPUT_AUTOFLUSH
$|
If set to nonzero, forces a flush right away and after every write or
print on the currently selected output channel. Default is 0
(regardless of whether the channel is really buffered by the system or
not; $| tells you only whether you've asked Perl explicitly to flush
after each write). STDOUT will typically be line buffered if output is
to the terminal and block buffered otherwise. Setting this variable is
useful primarily when you are outputting to a pipe or socket, such as
when you are running a Perl program under rsh and want to see the
output as it's happening.
See the article Suffering from Buffering? for more details.
Every time when I finish running a matlab code collection on command line, when I exit matlab, the standard output just gets messed. I can still use the terminal window, but whatever I typed won't show up on the screen, leaving me either type with my eyes blind, or open up a new terminal and excessively cd to the old place.
This happens every single time when I use make to run a matlab collection, and since I'm working a lot on this, it turns out to be very annoying. Does anyone know what's the problem here and how should I fix it?
As was pointed out in the comments, the makescript is probably dumping "bad" characters to the terminal. You could prevent this (but possibly lose useful information) by redirecting the output - instead of sending it to the terminal window, you can send it to a file, or even /dev/null ("the great bit bucket in the sky").
The underlying problem, however, is that your makefile is even sending these characters to the terminal in the first place. I would recommend that you pipe the output to a file with something like make > myDump.txt, then examine the resulting file to see what is going on, and where in your makefile the problem is created. It is possible that you will still be getting some output when you do this - that's because by default > redirects stdout only, and not stderr - a second output stream used for error messages. You can redirect both to a file with make 2>&1 myDump.txt.
You have already seen the recommendation to use stty sane to restore the status of the terminal - I am repeating it here in case someone only looks at answers, and not at comments; but I don't take credit for it :-).
Got stuck with one problem in our live server.
Have script (perl) which runs almost 15 to 18 hrs a day. it creates 100+ sub process every day . One place it has command (product command which we run in command line solaris box) which is being triggerred with back ticks inside perl code.
It looks like the back ticks command gets skipped or failed randomly.
for eg. if i need to run for 50 customers 2 or 3 gets failed randomly.
I do not see the evidence that the command has been triggerred in anywhere.
since its live server we can't even try making much in code change until we are sure about the problem.
here is the code..
my $comm = "inventory -noX customer1"; #sample command i have given here
my $newLogFile = "To capture command output here we have path whre the file gets created");
my $piddy = `$comm 2>&1 > $newLogFile`;
Is it because of the back ticks it happens I am really not sure :(.
Also tried various analysis like memory/CPU/diskspace/Adding librtld_db.so in LD_LIBRARY_PATH etc....but no luck...Also the perl is in 64 bit ...what else Can i? :(
I suspect you are not checking for errors (and perl doesn't make that easy to do correctly for backticks).
Consider using IPC::System::Simple's capture in place of your backticks/qx.
As its doc says, "If there's an error, it will die with a detailed description of what went wrong."
It shouldn't fail just because of backticks, however because it is spawning a new process, that process may be periodically subject to failure due to system conditions (eg. sysLoad). Backticks are really a "fire and forget" method and should never be used for anything critical in a production environment. As previously suggested, there are far more detailed ways to manage spawning external processes.
If the command's output is being lost due to buffering, you might try turning off buffering, but keep an eye on it for performance degradation (it's usually not significant).
Buffering can be turned off for an entire script by adding this near the top:
$|=1;
When calling external commands, I'm using system of IPC::System::Simple or open3 of IPC::Open3.
I have Perl program based on IO::Async, and it sometimes just exits after a few hours/days without printing any error message whatsoever. There's nothing in dmesg or /var/log either. STDOUT/STDERR are both autoflush(1) so data shouldn't be lost in buffers. It doesn't actually exit from IO::Async::Loop->loop_forever - print I put there just to make sure of that never gets triggered.
Now one way would be to keep peppering the program with more and more prints and hope one of them gives me some clue. Is there better way to get information what was going on in a program that made it exit/silently crash?
One trick I've used is to run the program under strace or ltrace (or attach to the process using strace). Naturally that was under Linux. Under other operating systems you'd use ktrace or dtrace or whatever is appropriate.
A trick I've used for programs which only exhibit sparse issues over days or week and then only over handfuls among hundreds of systems is to direct the output from my tracer to a FIFO, and have a custom program keep only 10K lines in a ring buffer (and with a handler on SIGPIPE and SIGHUP to dump the current buffer contents into a file. (It's a simple program, but I don't have a copy handy and I'm not going to re-write it tonight; my copy was written for internal use and is owned by a former employer).
The ring buffer allows the program to run indefinitely with fear of running systems out of disk space ... we usually only need a few hundred, even a couple thousand lines of the trace in such matters.
If you are capturing STDERR, you could start the program as perl -MCarp::Always foo_prog. Carp::Always forces a stack trace on all errors.
A sudden exit without any error message is possibly a SIGPIPE. Traditionally SIGPIPE is used to stop things like the cat command in the following pipeline:
cat file | head -10
It doesn't usually result in anything being printed either by libc or perl to indicate what happened.
Since in an IO::Async-based program you'd not want to silently exit on SIGPIPE, my suggestion would be to put somewhere in the main file of the program a line something like
$SIG{PIPE} = sub { die "Aborting on SIGPIPE\n" };
which will at least alert you to this fact. If instead you use Carp::croak without the \n you might even be lucky enough to get the file/line number of the syswrite, etc... that caused the SIGPIPE.