Perl hangs when pushing onto array - perl

I'm running the following code to extract a list of IDs from a mysql database and store them in an array. I'm doing this exact sequence of steps in other places in my code, but in this location, something funny happens and Perl just hangs and never returns. Every where else in the code, it works fine. If I run this code as is, it runs fine, but if I uncomment the push() function, it hangs. Is this a bug?
my $result = $db->query($sqlstring)
if ($result->numrows > 0) {
my #list = ();
while (my %row = $result->fetchhash) {
my $studyid = $row{'study_id'} + 0;
print "$studyid\n";
WriteLog("Found study [" . $studyid . "]");
#push(#list,$studyid); # uncomment this to hang it
}
return \#list;
}
EDIT: I tried the data::dumper. But now the freezing occurs in a new location. In the code below, it freezes right after the very last Dumper(#list) statement. (I also switched to DBI from MySQL module, but that had no effect).
my #list = ();
my $result = $dbh->prepare($sqlstring);
$result->execute();
WriteLog($sqlstring);
if ($result->rows > 0) {
while (my $row = $result->fetchrow_hashref()) {
my $studyid = $row->{study_id};
WriteLog("Found study [" . $studyid . "]");
push #list,$studyid;
}
}
print Dumper(#list);
return \#list;

When is this freezing? Is it freezing on the first call to push, or on some subsequent call?
Are you sure you're pushing what you think you're pushing?
What you should do is to use Data::Dumper; and then do a dumper before the push.
use Data::Dumper;
use feature qw(say);
my $result = $db->query($sqlstring)
if ( $result->numrows > 0 ) {
my #list;
while ( my %row = $result->fetchhash ) {
my $studyid = $row{study_id} + 0;
say $studyid;
WriteLog("Found study [" . $studyid . "]");
say "List: " . Dumper #list;
say "Study ID: " . Dumper \$studyid;
push #list, $studyid; # uncomment this to hang it
}
return \#list;
}
Note I'm using say instead of print. Dumper can cause problems if you don't parenthesize it and use print. If you don't want to use say, then you'll need to do this:
print "Study ID: " . Dumper ( \$studyid ) . "\n";
By the way, what do you return if there are no rows?

I was going to mention that we just had this same problem with one of our scripts. The script is being run through Jenkins and the problem was that there wasn't really a problem. We just hadn't seen all of the output up to the point of the program's current state. So it looked like it was hung on a push - it was actually hung legitimately on a sleep() call a few lines later. It was fixed with the auto-flush flag which David W. mentioned. The problem was worsened by the use of Jenkins as an execution environment (vs the command-line debugger). The STDOUT/STDIN pipe opened in the Jenkins context doesn't exist in a command-line shell. So we would never see the issue from the command line - further confusing us. Once I realized that there was a communications pipe involved then things fell into place.

Related

<STDIN> not working during while - Perl

I wrote a small script, attempting to write a calculator, just to test what I know, but I have a problem, where I want to read <STDIN> in a while loop, and it just freezes till I kill it with CTRL + C.
The weird thing, is that it works in the eclipse console, but nowhere else.
Relevant code:
require Term::Screen::Uni;
use strict;
...
sub basicCalc(){
my $scr = new Term::Screen::Uni;
$scr->clrscr();
print("Basic calculator - type help for information about your options!\r\n");
while($input == "1\n"){
bcOPR();
}
$scr->clrscr();
main();
}
sub bcOPR(){
my $inp = <>;
if($inp eq "help\n"){ $input = 0; }
}

Looping through data provided by Net::SSH2

I am using the following lib Net::SSH2
I can connect to my device and get the output OK in most case. Below is the relevant code:
sub logIntoDevice
{
my $self = shift;
my ($ssh2) = #_;
if(! $ssh2->connect($self->deviceIP))
{
say "Failed to connect to device:",$self->deviceIP;
$ssh2->disconnect();
exit 0;
}
if(! $ssh2->auth_password($self->usr, $self->pass))
{
say "Authentication Fail to ",$self->deviceIP;
$ssh2->disconnect();
exit 0;
}
my $channel = $ssh2->channel();
$channel->blocking(0);
$channel->shell();
return $channel;
}
sub sendCmd
{
my $self = shift;
my ($cmd,$channel) = #_;
my #cmdOutput;
print $channel "$cmd\n";
while (<$channel>)
{
chomp $_;
push(#cmdOutput, $_);
}
return #cmdOutput;
}
So below are the cmd i sent to the sub's. They work fine and the output is write to file OK.
$self->writeToFile($fileName,$self->sendCmd("show clock",$channel));
$self->writeToFile($fileName,$self->sendCmd("\n",$channel));
$self->writeToFile($fileName,$self->sendCmd("dir",$channel));
with the exception of when i sent the following cmd:
$self->writeToFile($fileName,$self->sendCmd("sh run",$channel));
the output of the cmd on the device using putty is:
sh run
Building configuration...
Current configuration : 16575 bytes
!
!
!Last configuration change at 16:37:19 CET+1 Sat Mar 15 2014
.....
but in the log file all you see is
sh run
Building configuration...
so the issue is the blank lines after the Building configuration output make the while (<$channel>) think its the end of the output.
My issue is I cant figure a way to loop through the data without using a While loop.
UPDATE
Ok come up with this solution but seems very clunky. must be a better way if doing this
sub sendCmd
{
my $self = shift;
my ($cmd,$channel) = #_;
my #cmdOutput;
my $currPrompt;
#get prompt. i am sure there is a better way!!! just cant figure it out
print $channel "\n";
while (<$channel>)
{
$currPrompt = $_;
}
print $channel "$cmd\n";
while(42)
{
my $inerOutput;
while (<$channel>)
{
chomp $_;
$inerOutput = $_;
push(#cmdOutput, $_);
}
if($inerOutput ne $currPrompt)
{
sleep(7);
}
else
{
last;
}
}
return #cmdOutput;
}
I don't think your issue is blank lines. Most likely, the issue is that you are using non-blocking mode, and it takes time for the device to perform the command. So you are getting an empty line (or an undef) after "Building configuration..." is read because there is no extra output produced yet.
I would use Net::SSH2's poll method with a timeout, which will let you know when there is something to read. If "sh run" takes substantially longer than other commands you are issuing, your sendCmd method needs to be aware of this, and allow more time to pass before it decides no more output is coming its way.
Alternatively, you can (as is the custom when using, for example, Net::Telnet) wait for more output until you see the prompt, whatever the prompt is for the device in question,
and then you will know that the command has finished its execution.
Net::SSH2->poll is deprecated as result of libss2_poll deprecation

Perl FCGI Exit Without Dieing

How to end script without using using exit if using Perl FCGI. After searching for days the only solution I found is to jump at label in the main script. below is the code of the main index.fcgi.
$fcgi_requests = 0; # the number of requests this fcgi process handled.
$handling_request = 0;
$exit_requested = 0;
$app_quit_request = 0; # End the application but not the FCGI process
# workaround for known bug in libfcgi
while (($ignore) = each %ENV) { }
$fcgi_request = FCGI::Request();
#$fcgi_request = FCGI::Request( \*STDIN, \*STDOUT, \*STDERR, \%ENV, $socket);
sub sig_handler {
my ($callpackage, $callfile, $callline) = caller;
if ($app_quit_request) {
$app_quit_request = 0;
goto ABORTLABEL;
}
$exit_requested = 1;
exit(0) if !$handling_request;
}
$SIG{USR1} = \&sig_handler;
$SIG{TERM} = \&sig_handler;
$SIG{PIPE} = 'IGNORE';
#The goal of fast cgi is to load the program once, and iterate in a loop for every request.
while ($handling_request = ($fcgi_request->Accept() >= 0)) {
process_fcgi_request();
$handling_request = 0;
last if $exit_requested;
#exit if -M $ENV{SCRIPT_FILENAME} < 0; # Autorestart
}
$fcgi_request->Finish();
exit(0);
#=========================================================#
sub process_fcgi_request() {
$fcgi_requests++;
# dispatch current request
my_app();
$fcgi_request->Finish();
}
#=========================================================#
# let it think we are done, used by abort
ABORTLABEL:
$fcgi_request->Finish();
#=========================================================#
The main request is I want to stop the program execution from inside sub insidi modules that may be called by long depth for example inside a login function in a accounts module.
Of course I can not use exit because it will terminate the fcgi process, I tried all error and throw and try modules all use die which also ends the process. Of course I can use the return from each sub but this will require to rewrite the whole program for fcgi.
The normal way to model exceptions in Perl is to call die inside eval BLOCK, which catches the die and so doesn't terminate the process. It'll just terminate the eval and the program continues to run from immediately afterwards. As far as I've seen, the exception-handling modules on CPAN are mostly wrappers around this basic functionality to give it different syntax or make it easier to write catch blocks. Therefore I'm surprised these don't work for you. Did you actually try them or did you just assume die always kills the process? The name is slightly misleading, because it really means 'throw an exception'. Just if you do that outside an eval the interpreter catches it, and its only response is to terminate the process.
eval {
say "Hello world";
die;
say "Not printed";
};
say "Is printed";
You don't want to call exit inside an eval though. Nothing catches that.
I would recommend though rewriting the entire control flow for FCGI. The lifecycle of your code changes significantly, so you have to make a certain amount of modifications to make sure that variable re-use is working properly and you're not leaking memory. Often it's better to do that up front rather than spend days tracking down odd bugs later.
After several questions and deep research, I got this solution. This coding example allows you to return from any nested levels of calls. The module Scope::Upper is XS so it should be fast.
use Scope::Upper qw/unwind CALLER/;
sub level1 {
print "before level 1 \n";
level2();
print "after level 1 \n";
}
sub level2 {
print "before level 2 \n";
level3();
print "after level 2 \n";
}
sub level3 {
print "before level 3 \n";
level4();
print "after level 3 \n";
}
sub level4 {
print "before level 4 \n";
#unwind CALLER 2;
my #frame;
my $i;
#$i++ while #frame = caller($i);# and $frame[0] ne "main";
$i++ while #frame = caller($i);
#print "i=: $i \n";
#unwind CALLER (#frame ? $i : $i - 1);
unwind CALLER $i-1;
print "after level 4 \n";
}
print level1();
If you run this code the output will be:
before level 1
before level 2
before level 3
before level 4
You can return to any up level using:
my intLevel = 2;
unwind CALLER intLevel;

just can't get perl working as expected ( conditionals and variable declaring )

EDIT:
I will try a better explication this time, this is the exact code from my script (sorry for all them coments, they are a result of your sugestions, and apear in the video below).
#use warnings;
#use Data::Dumper;
open(my $tmp_file, ">>", "/tmp/some_bad.log") or die "Can not open log file: $!\n";
#if( $id_client != "")
#allowed_locations = ();
#print $tmp_file "Before the if: ". Data::Dumper->Dump([\#allowed_locations, $id_client]) . "";
if( $id_client )
{
# print $tmp_file "Start the if: ". Data::Dumper->Dump([\#allowed_locations, $id_client]) . "";
# my $q = "select distinct id_location from locations inner join address using (id_db5_address) inner join zona_rural_detaliat using (id_city) where id_client=$id_client";
# my $st = &sql_special_transaction($sql_local_host, $sql_local_database, $sql_local_root, $sql_local_root_password, $q);
# print $tmp_file "Before the while loop: ref(st)='". ref($st) . "\n";
# while((my $id)=$st->fetchrow())
# {
# print $tmp_file "Row the while loop: ". Data::Dumper->Dump([$id]) . "";
# my $id = 12121212;
# push(#allowed_locations, $id);
# }
# print $tmp_file "After the while loop: ref(st)='". ref($st) . "\n";
# my($a) = 1;
#} else {
# my($a) = 0;
}
#print $tmp_file "After the if: ". Data::Dumper->Dump([\#allowed_locations, $id_client]) . "";
close($tmp_file) or die "Can not close file: $!\n";
#&html_error(#allowed_locations);
First off all, somebody said that I should try to run it in command line, the script works fine in command line (no warnings, It was uncommented then), but when triyng to load in via apache in the browser it fails, please see this video where I captured the script behavior, what I tried to show in the video:
I have opened 2 tabs the first doesn't define the variable $id_client, the second defines the variable $id_client that is read from GET: ?id_client=36124 => $id_client = 36124; , both of them include the library in the video "locallib.pl"
When running the script with all the
new code commented the page loads
when uncoment the line that defines
the #allowed_locations = (); the
script fails
leave this definition and uncoment
the if block, and the definition of
my $a; in the if block; Now the script works fine when $id_client is
defined, but fails when $id_client
is not defined
Uncoment the else block and the
definition of my $a; in the else
block. Now the script works fine
with or without $id_client
now comment all the my $a;
definisions and comment the else
block, the script fails
but if I'm using open() to open
a file before the IF, and
close() to close it after the if it does't fail even if the IF block
is empty and event if there is no
else block
I have replicated all the steps when running the script in the command line, and the script worked after each step.
I know it sounds like something that cannot be the behavior of the script, but please watch the video (2 minutes), maybe you will notice something that I'm doing wrong there.
Using perl version:
[root#db]# perl -v
This is perl, v5.8.6 built for i386-linux-thread-mult
Somebody asked if I don't have a test server, answer: NO, my company has a production server that has multiple purposes, not only the web interface, and I cannot risk to update the kernel or the perl version, and cannot risk instaling any debuger, as the company owners say: "If it works, leave it alone", and for them the solution with my ($a); is perfect beacause it works, I'm asking here just for me, to learn more about perl, and to understand what is going wrong and what can I do better next time.
Thank you.
P.S. hope this new approach will restore some of my -1 :)
EDIT:
I had success starting the error logging, and found this in the error log after each step that resulted in a failure I got this messages:
[Thu Jul 15 14:29:19 2010] [error] locallib.pl did not return a true value at /var/www/html/rdsdb4/cgi-bin/clients/quicksearch.cgi line 2.
[Thu Jul 15 14:29:19 2010] [error] Premature end of script headers: quicksearch.cgi
What I found is that this code is at the end of the main code in the locallib.pl after this there are sub definitions, and locallib.pl is a library not a program file, so it's last statement must returns true. , a simple 1; statement at the end of the library ensures that (I put it after sub definitions to ensure that noobody writes code in the main after the 1;) and the problem was fixed.
Don't know why in CLI it had no problem ...
Maybe I will get a lot of down votes now ( be gentle :) ) , but what can I do ...and I hope that some newbies will read this and learn something from my mistake.
Thank you all for your help.
You need to explicitly check for definedness.
If you want to enter the loop when $client is defined,
use if ( defined $client ).
If you want to enter the loop when $client is defined and a valid integer,
use if ( defined $client && $client =~ /^-?\d+$/ ).
I assume it's an integer from the context, if it can be a float, the regex needs to be enhanced - there's a standard Perl library containing pre-canned regexes, including ones to match floats. If you require a non-negative int, drop -? from regex's start.
If you want to enter the loop when $client is defined and a non-zero (and assuming it shouldn't ever be an empty string),
use if ( $client ).
If you want to enter the loop when $client is defined and a valid non-zero int,
use if ( $client && $client =~ /^-?\d+$/ ).
Your #ids is "undef" when if condition is false, which may break the code later on if it relies on #ids being an array. Since you didn't actually specify how the script breaks without an else, this is the most likely cause.
Please see if this version works (use whichever "if" condition from above you need, I picked the last one as it appears to match the closest witrh the original code's intent - only enter for non-zero integers):
UPDATED CODE WITH DEBUGGING
use Data::Dumper;
open(my $tmp_file, ">", "/tmp/some_bad.log") or die "Can not open log file: $!\n";
#ids = (); # Do this first so #ids is always an array, even for non-client!
print $tmp_file "Before the if: ". Data::Dumper->Dump([\#ids, $client]) . "\n";
if ( $client && $client =~ /^-?\d+$/ ) # First expression catches undef and zero
{
print $tmp_file "Start the if: ". Data::Dumper->Dump([\#ids, $client]) . "\n";
my $st = &sql_query("select id from table where client=$client");
print $tmp_file "Before the while loop: ref(st)='". ref($st) . "'\n";
while(my $row = $st->fetchrow())
{
print $tmp_file "Row the while loop: ". Data::Dumper->Dump([row]) . "'\n";
push(#ids, $row->[0]);
}
print $tmp_file "After the while loop: ref(st)='". ref($st) . "'\n";
# No need to undef since both variables are lexically in this block only
}
print $tmp_file "After the if\n";
close($tmp_file) or die "Can not close file: $!\n";
when checking against a string, == and != should be respectively 'eq' or 'ne'
if( $client != "" )
should be
if( $client ne "" )
Otherwise you don't get what you're expecting to get.
Always begin your script with :
use warnings;
use strict;
these will give you usefull informations.
Then you could write :
my #ids;
if (defined $client) {
#ids = (); # not necessary if you run this part only once
my $st = sql_query("select id from table where client=$client");
while( my ($id) = $st->fetchrow ) {
push #ids, $id;
}
} else {
warn '$client not defined';
}
if (#ids) { # Your query returned something
# do stuff with #ids
} else {
warn "client '$client' does not exist in database";
}
Note: this answer was deleted because I consider that this is not a real question. I am undeleting it to save other people repeating this.
Instead of
if( $client != "" )
try
if ($client)
Also, Perl debugging is easier if you
use warnings;
use strict;
What I found is that this code is at the end of the main code in the locallib.pl after this there are sub definitions, and locallib.pl is a library not a program file, so it's last statement must returns true, a simple 1; statement at the end of the library ensures that (put it after sub definitions to ensure that noobody writes code in the main after the 1;) and the problem was fixed.
The conclusion:
I have learned that every time you write a library or modify one, ensure that it's last statment returns true;
Oh my... Try this as an example instead...
# Move the logic into a subroutine
# Forward definition so perl knows func exists
sub getClientIds($);
# Call subroutine to find id's - defined later.
my #ids_from_database = &getClientIds("Joe Smith");
# If sub returned an empty list () then variable will be false.
# Otherwise, print each ID we found.
if (#ids_from_database) {
foreach my $i (#ids_from_database) {
print "Found ID $i \n";
}
} else {
print "Found nothing! \n";
}
# This is the end of the "main" code - now we define the logic.
# Here's the real work
sub getClientIds($) {
my $client = shift #_; # assign first parameter to var $client
my #ids = (); # what we will return
# ensure we weren't called with &getClientIds("") or something...
if (not $client) {
print "I really need you to give me a parameter...\n";
return #ids;
}
# I'm assuming the query is string based, so probably need to put it
# inside \"quotes\"
my $st = &sql_query("select id from table where client=\"$client\"");
# Did sql_query() fail?
if (not $st) {
print "Oops someone made a problem in the SQL...\n";
return #ids;
}
my #result;
# Returns a list, so putting it in a list and then pulling the first element
# in two steps instead of one.
while (#result = $st->fetchrow()) {
push #ids, $result[0];
}
# Always a good idea to clean up once you're done.
$st->finish();
return #ids;
}
To your specific questions:
If you want to test if $client is defined, you want "if ( eval { defined $client; } )", but that's almost certainly NOT what you're looking for! It's far easier to ensure $client has some definition early in the program (e.g. $client = "";). Also note Kaklon's answer about the difference between ne and !=
if (X) { stuff } else { } is not valid perl. You could do: if (X) { stuff } else { 1; } but that's kind of begging the question, because the real issue is the test of the variable, not an else clause.
Sorry, no clue on that - I think the problem's elsewhere.
I also echo Kinopiko in recommending you add "use strict;" at the start of your program. That means that any $variable #that %you use has to be pre-defined as "my $varable; my #that; my %you;" It may seem like more work, but it's less work than trying to deal with undefined versus defined variables in code. It's a good habit to get into.
Note that my variables only live within the squiggliez in which they are defined (there's implicit squiggliez around the whole file:
my $x = 1;
if ($x == 1)
{
my $x = 2;
print "$x \n"; # prints 2. This is NOT the same $x as was set to 1 above.
}
print "$x \n"; # prints 1, because the $x in the squiggliez is gone.

Why are some characters missing when I converted my Perl script to executable using Perlapp?

Before posting my question to the ActiveState forum, I'd like to try luck here :)
I'm trying to convert a simple script of mine to .exe file using Perlapp (version 8.1). The Perl script works fine and it seems Perlapp also did its job successfully.
But the converted .exe file has some weird behavior, which, I believe, must be related to utf-8 encoding. For example, the Perl script would yield the result like:
hàn huáng zhòng sè sī qīng guó
But running the executable file would give me only this:
h hu zh s s q gu
I've already configured Perlapp so that utf8.pm is explicitly added but the problem just refuses to go away. I've tried something else. For example,
binmode DATA, ":utf8";
and
">:encoding(utf8)"
but without any luck;
Can anyone kindly give me some hint as to what might be the reason? Thanks like always :)
I can post the whole code here but it seems unnecessary so I just paste some snippets of the code that I think is relevant to the problem:
use utf8;
%zidian = map {chomp;split/\s+/,$_,2} <DATA>;
open my $in,'<:utf8',"./original.txt";
open my $out,'>:utf8',"./modified.txt";
if ( $code~~%zidian) {
$value = lc$zidian{$code};
}
__DATA__
3400 Qiū
3401 TIǎN
3404 KUà
3405 Wǔ
And one more thing, I'm running ActivePerl 5.10.0.on Windows XP (Chinese Version) and the script is saved as utf-8 encoding without BOM. PerlApp cannot handle a script that has BOM.
Edit
If I were to give a workable snippet, then I suppose it's like giving the whole code because I'm using three inter-connected sub-routines, which I take with some modifications from Lingua::han::Pinyin module and Lingua::han::Utils module.
#! perl
# to make good vertical alignment,
# set font family to SonTi and font size to Four(12pts)
use utf8;
sub Unihan {
my $hanzi = shift;
my #unihan = map { uc sprintf("%x",$_) } unpack ("U*", $hanzi);
}
sub csplit {
my $hanzi = shift;
my #return_hanzi;
my #code = Unihan($hanzi);
foreach my $code (#code) {
my $value = pack("U*", hex $code);
push #return_hanzi, $value if ($value);
}
return wantarray ? #return_hanzi : join( '', #return_hanzi );
}
%zidian = map {chomp;split/\s+/,$_,2} <DATA>;
sub han2pinyin {
my $hanzi = shift;
my #pinyin;
my #code = Unihan($hanzi);
foreach $code (#code) {
if ( $code~~%zidian) {
$value = lc$zidian{$code};
}
else {
$value = " ";
}
push #pinyin, $value;
}
return wantarray ? #pinyin : join( '', #pinyin );
}
open $in,'<:utf8',"./original.txt";
seek $in, 3,0;
open $out,'>:utf8',"./modified.txt";
while(<$in>){
s/(.{18})/$1\n/g;
push #tmp, $_;
}
foreach (#tmp){
my #hanzi;
my #pinyin;
#hanzi = csplit($_);
my $hang = join "", #hanzi;
#pinyin = han2pinyin($hang);
for ( my $i = 0; $i < #hanzi && $i < #pinyin; ++$i ) {
if ( $hanzi[$i] =~ /[\xEFBC8C]|[\xE38082]|[\xEFBC81]|[\xEFBC9F]|[\xE2809C]|[\xE2809D]|[\xEFBC9A]/ ) {
splice(#pinyin, $i, 0," ");
}
}
printf $out "%-7s" x #pinyin, #pinyin;
print $out "\n";
printf $out "%-6s" x #hanzi, #hanzi;
print $out "\n";
}
__DATA__
3400 Qiū
3401 TIǎN
3404 KUà
3405 Wǔ
ActiveState hasn't given me any help yet. Whatever. Now I've figured out a workaround for my problem and this workaround looks very weird.
First I added some otherwise useless utf-8 encoded characters to my DATA section, like the following:
__DATA__
aardvark 'ɑ:dvɑ:k
aardwolf 'ɑ:dwulf
aasvogel 'ɑ:sfәugәl
3400 Qiū
3401 TIǎN
3404 KUà
3405 Wǔ
And then I removed the use utf8; pragma from my script;
and then I removed the utf8 flag from the following line of code:
open $out,'>:utf8',"./modified.txt";
Now it becomes
open $out,'>',"./modified.txt";
But I had to let the following line of code unchanged:
open $in,'<:utf8',"./original.txt";
Then everything was okay except that I'd receive "wide characters in print" warnings. But I added another line of code:
no warnings;
And then I Perlapped my script and everything worked fine :)
This is really strange. I'm suspecting this problem is somehow OS specific. It's also quite likely that there's something wrong with my Windows system. And I also tried Perl2exe and the compiled executable gave me some "memory 0010c4 cannot be read" error. Whatever. My problem is somehow fixed by myself :)