I'm running a perl script to pull a list of about 20 text files, and parse through them. For some reason my process is bombing partway through the list, and am having trouble debugging it.
Anyone know the location of the Strawberry perl log file, and if there's a builtin max execution time, or memory limit variable like in PHP?
There are three files:
1. cron.php
2. nightly_script.php
3. myscript.pl
It successfully executes the first insert statement in that while loop, but not anymore after that. Since this is running like a cron job I don't have any output window to look at. This is why I was hoping there's a log somewhere, so if there's a syntax error, or a mysql error I can see it somewhere. Also, if I just run myscript.pl on the file in question directly, it works no problem.
cron.php
date_default_timezone_set('America/New_York');
/*
min hr dom month dow cmd
hour in 24 hour format, no leading zeros
*/
$jobsQueue = Array();
$jobsQueue[] = Array('10', '0', '*', '*', '*', 'php c:\nightly_script.php'); // These items are order dependent, so run as one script that synchronously executes each command
while(1) {
$now = time();
$min = date('i',$now);
$hr = date('G',$now);
echo "$hr:$min\n";
foreach($jobsQueue AS $job) {
if($job[0] == $min && $job[1] == $hr) {
system("$job[5]>NULL");
}
}
sleep(60);
}
?>
nightly_script.php
// Process Hand Histories
system('perl myscript.pl');
?>
myscript snippet
while ( ($key, $value) = each(%players) ) {
print "$key => $value\n";
if($value > 0)
{
$uname = $key;
$uname =~ s/player(.*)(\s*)/$1/;
$connect = DBI->connect("DBI:mysql:database=$config_mysql_db;host=$config_mysql_server",$config_mysql_user,$config_mysql_pass,{'RaiseError' => 1});
print "\n*****\n$uname\n*****\n";
$updateStatement = "INSERT statement";
$executeStatement = $connect->prepare($updateStatement);
$executeStatement->execute();
$updateStatement = "UPDATE command";
$executeStatement = $connect->prepare($updateStatement);
$executeStatement->execute();
delete $players{$key};
# Clean up the record set and the database connection
$connect->disconnect();
}
elsif($value <= 0)
{
delete $players{$key};
}
}
Since perl doesn't have a log like php, you can create your own log file by redirecting perl's stdout and stderr to a file. Try doing this by modifying the system call in nightly_script.php.
system('perl myscript.pl 1>myperllog.txt 2>&1');
or
system('perl myscript.pl 1>myperllog.txt 2>myperllog.err');
Related
Problem Statement --
I display a message to user using Perl and takes an input.On the basis of input I decide whether I need to do further processing or not.This processing takes a long time ( say 5 hour) and user run this process by logging into remote Unix/Linux system.Hence to make sure that network malfunctioning will not affect the process; I want to switch the process to background.
How can I switch such running Perl process to background?
or
Is it possible to take user input from current terminal( the terminal from where user run process as input need to be taken at very starting) if process is running into background?
OS - Linux variants
Yup, you want to daemonize your program after it finishes its interaction with the user. I would encourage you to use a module like Proc::Daemon to do the work, though: there are a bunch of subtleties in doing it correctly. The POD for Proc::Daemon gives a good description of its usage, but a simple usage can be as basic as
use Proc::Daemon;
# ... finished the interactive stuff
my $pid = Proc::Daemon::Init( { work_dir => '/var/run/my_program' })
exit 0 if ($pid == 0);
die "Error daemonizing, cannot continue: $!\n" if ($! != 0);
# ... now do the background processing
# note that STDOUT and STDERR are no longer connected to the user's terminal!
Here is a very, very simple example for my comment above...
#!/usr/bin/perl
use strict;
use warnings;
my $lcnt = 0;
if( !$ARGV[0] ) { # If no ARGS on the command line, get user input
print "How many lines do you want to print?";
chomp( $lcnt = <STDIN> );
if( $lcnt > 0 ) {
# when we are sure we have what we need
# call myself.pl and put it in the background with '&'
my $cmd = "./myself.pl ".$lcnt.' &';
system($cmd);
exit(0);
} else { die "Invalid input!\n"; }
} else { # Otherwise, lets do the processing
$lcnt = $ARGV[0];
for( my $x = 0; $x <= $lcnt; $x++ ) {
my $cmd = "echo 'Printing line: $lcnt' >> /tmp/myself.txt";
system($cmd);
sleep(1);
}
}
exit(0);
If you save this to a file called 'myself.pl' then run it. With no arguments on the command line, the script will ask you to input a number. Type in 20 and press enter. You'll see the script exit almost instantly. But if you quickly
tail -f /tmp/myself.txt
you'll see that the background process is still running, printing a new line to the file every second. Also, typing the 'ps' command on Linux systems, should show the spawned process running in the background:
jlb#linux-f7r2:~/test> ps
PID TTY TIME CMD
1243 pts/1 00:00:00 bash
4171 pts/1 00:00:00 myself.pl
4176 pts/1 00:00:00 ps
Demonise the process if the correct input:
#test input
if($inputsuccess) {
if(fork() = 0) {
#child
if(fork() = 0) {
#child
#background processing
}
} else {
wait();
}
}
I've got a script that reformats incoming data from a CSV into a readable format by a vended system. I may be going crazy, but I'm pretty sure it worked a week or two ago in the production environment. However, at some point in the last week or two, it stopped working. I tracked the problem down to the Text::CSV module not populating the $csv->fields() property.
my $csv = Text::CSV->new({sep_char => '|', allow_loose_quotes => 1});
$csv->column_names($csv->getline(*READ));
my #keys = $csv->fields;
Now, on my local machine (and, at least in my head, in the production environment two weeks ago, too), this would populate #keys with the parsed header fields. However, now, in both production and pre-production, this fails. The only difference I can tell is that my machine is running perl 5.12.4, while the prod/pprd is 5.8.8. The Text::CSV module on both is 1.21.
On my machine, if I use Data::Dumper and dump the $csv object, part of the properties is
'_FIELDS' => [
'ID',
'IDCARD_TYPE',
'FIRST_NAME',
'MIDDLE_NAME',
'LAST_NAME',
...
'EMAIL',
],
On the other machines:
'_FIELDS' => undef,
I've worked around this by using $csv->column_names to populate #keys, but something doesn't seem right and I'd really like to figure out what's going on. Any ideas?
Per the Text::CSV documentation, returning undef is the expected result of fields() after calling getline(). Try using parse() first. You might be using a different version of this module on your local machine. You can check the version using perl -MText::CSV -e 'print $Text::CSV::VERSION'.
Note that the return value is undefined after using getline (), which
does not fill the data structures returned by parse ().
Following alternate sequence worked for me:
$file = "test.csv" ;
if(!open($fh, "<", $file )) {
# Cannot call getline is a symptom of a bad open()
printf("### Error %s: could not open file %s\n", $ws, $file) ;
close($fh) ;
exit 1 ;
}
while(my $row = $csv->getline($fh)) {
# $row is a pointer to an Array
# The array is already parsed.
#items = #{$row} ;
for(my $i=0 ; $i<=$#items; $i++) {
printf("Field %d: (%s)\n", $i, $items[$i] ) ;
}
}
close $fh ;
Friends...
I have got bash script which calls perl script and emails logfile result everytime.
I want to change my bash script such that it should only email if there is value in perl subroutine row counter (rcounter++) and not all time.
any tips on how to change .ksh file?
.ksh
#!/bin/ksh
d=`date +%Y%m%d`
log_dir=$HOME
output_file=log.list
if ! list_tables -login /#testdb -outputFile $output_file
then
mailx -s "list report : $d" test#mail < $output_file
fi
=======Below if condition also works for me=============================
list_tables -login /#testdb -outputFile $output_file
if ["$?" -ne "0"];
then
mailx -s "list report : $d" test#mail < $output_file
fi
========================================================================
Perl Script: list_tables
use strict;
use Getopt::Long;
use DBI;
use DBD::Oracle qw(:ora_types);
my $exitStatus = 0;
my %options = ()
my $oracleLogin;
my $outputFile;
my $runDate;
my $logFile;
my $rcounter;
($oracleLogin, $outputFile) = &validateCommandLine();
my $db = &attemptconnect($oracleLogin);
&reportListTables($outputFile);
$db->$disconnect;
exit($rcounter);
#---------------------------
sub reportListTables {
my $outputFile = shift;
if ( ! open (OUT,">" . $outputfile)) {
&logMessage("Error opening $outputFile");
}
print OUT &putTitle;
my $oldDB="DEFAULT";
my $dbcounter = 0;
my $i;
print OUT &putHeader();
#iterate over results
for (my $i=0; $i<=$lstSessions; $i++) {
# print result row
print OUT &putRow($i);
$dbCounter++;
}
print OUT &putFooter($dbCounter);
print OUT " *** Report End \n";
closeOUT;
}
#------------------------------
sub putTitle {
my $title = qq{
List Tables: Yesterday
--------------
};
#------------------------------
sub putHeader {
my $header = qq{
TESTDB
==============
OWNER Table Created
};
#------------------------------
sub putRow {
my $indx = shift;
my $ln = sprintf "%-19s %-30s %-19s",
$lstSessions[$indx]{owner},
$lstSessions[$indx]{object_name},
$lstSessions[$indx]{created};
return "$ln\n";
}
#------------------------------
sub getListTables {
my $runDt = shift;
$rcounter = 0;
my $SQL = qq{
selct owner, object_name, to_char(created,'MM-DD-YYYY') from dba_objects
};
my $sth = $db->prepare ($SQL) or die $db->errstr;
$sth->execute() or die $db->errstr;;
while (my #row = $sth->fethcrow_array) {
$lstSessions[$rcounter] {owner} =$row[0];
$lstSessions[$rcounter] {object_name} =$row[1];
$lstSessions[$rcounter] {created} =$row[2];
&logMessage(" Owner: $lstSessions[$rcounter]{owner}");
&logMessage(" Table: $lstSessions[$rcounter]{object_name}");
&logMessage(" created: $lstSessions[$rcounter]{created}");
$rcounter++;
}
&logMessage("$rcounter records found...");
}
thanks..
also happy to include mail-x part in perl if that makes life more easy..
I am not sure I understood your question correctly. Also, your code is incomplete. So there's some guessing involved.
You cannot check the value of a local Perl variable from the caller's side.
But if your question is if the Perl code added anything to the logfile, the solution is simple: Delete the "rcounter records found..." line (which doesn't make sense anyway since it is always executed, whether the query returned results or not). Then, let the shell script backup the logfile before the call to Perl, and make a diff afterwards, sending the mail only if diff tells you there has been output added to the logfile.
If this doesn't help you, please clarify the question.
EDIT (from comments below):
Shell scripting isn't that difficult. Right now, your Perl script ends with:
$db->($exitStatus);
That is your exit code. You don't check that in your shell script anyway, so you could change it to something more useful, like the number of data rows written. A primitive solution would be to make $rcounter global (instead of local to getListTables()), by declaring it at the top of the Perl script (e.g. after my $logFile;). Then you could replace the "exitStatus" line above with simply:
$rcounter;
Voila, your Perl script now returns the number of data rows written.
In Perl, a return code of 0 is considered a failure, any other value is a success. In shell, it's the other way around - but luckily you don't have to worry about that as Perl knows that and "inverts" (negates) the return code of a script when returning to the calling shell.
So all you need is making the mailing depend on a non-zero return of Perl:
if list_tables -login /#testdb -outputFile $output_file
then
mailx -s "list report : $d" test#mail < $output_file
fi
A comment on the side: It looks to me as if your programming skill isn't up to par with the scope of the problem you are trying to solve. If returning a value from Perl to bash gives you that much trouble, you should probably spend your time with tutorials, not with getting input from a database and sending emails around. Learn to walk before you try to fly...
I'm having an issue with a Perl script relating to the Weather Research Forecast (WRF) model configuration. The script in question is a part of the download located here (login required, simple signup). If you download the most recent WRF-NMM core, in the unzipped directory is arch/Config_new.pl. The error that I'm having lies somewhere within lines 262-303:
until ( $validresponse ) {
print "------------------------------------------------------------------------\n" ;
print "Please select from among the following supported platforms.\n\n" ;
$opt = 1 ;
open CONFIGURE_DEFAULTS, "< ./arch/configure_new.defaults"
or die "Cannot open ./arch/configure_new.defaults for reading" ;
while ( <CONFIGURE_DEFAULTS> )
{
for $paropt ( #platforms )
{
if ( substr( $_, 0, 5 ) eq "#ARCH"
&& ( index( $_, $sw_os ) >= 0 ) && ( index( $_, $sw_mach ) >= 0 )
&& ( index($_, $paropt) >= 0 ) )
{
$optstr[$opt] = substr($_,6) ;
$optstr[$opt] =~ s/^[ ]*// ;
$optstr[$opt] =~ s/#.*$//g ;
chomp($optstr[$opt]) ;
$optstr[$opt] = $optstr[$opt]." (".$paropt.")" ;
if ( substr( $optstr[$opt], 0,4 ) ne "NULL" )
{
print " %2d. %s\n",$opt,$optstr[$opt] ;
$opt++ ;
}
}
}
}
close CONFIGURE_DEFAULTS ;
$opt -- ;
print "\nEnter selection [%d-%d] : ",1,$opt ;
$response = <STDIN> ;
if ( $response == -1 ) { exit ; }
if ( $response >= 1 && $response <= $opt )
{ $validresponse = 1 ; }
else
{ print("\nInvalid response (%d)\n",$response);}
}
Specifically, I am sent to an input line without any kind of prompting or list of what my options are. Only after I select a valid choice am I presented with the previous options. This is repeated a second time with another chunk of code further down (lines 478-528). What's got me confused is that, when I entered debugging mode, I inserted a break before the start of this portion of code. I ran p $validresponse and got the following:
0
If you REALLY want Grib2 output from WRF, modify the arch/Config_new.pl script.
Right now you are not getting the Jasper lib, from the environment, compiled into WRF.
This intrigues me, as the paragraph is from a printf from several lines before. In this particular script, it is the only printf that has run so far, but why the output was saved to the next created variable is beyond me. Any suggestions?
EDIT: After looking at choroba's suggestion, the same problem occurs with any type of redirection, whether piping, using tee, or stderr/stdout redirection. As such, I'm thinking it may be a problem with bash? That is, the only way I can run it is without any kind of logging (at least to my knowledge, which is admittedly quite limited).
You want to enable autoflushing, so that the Perl print buffer is flushed automatically after something is printed.
This is the default behavior when a Perl script outputs to a terminal window, but when the output is redirected in any way, the default is to buffer the output. Enabling autoflushing disables the buffering.
You can enable autoflushing by adding the following two lines to the top of the Perl script (below the Perl hashbang line, of course):
use IO::Handle qw();
STDOUT->autoflush(1);
When you redirect with pipes or similar you are (normally) redirecting STDOUT. All of the print statements go to STDOUT, so when redirecting the will be sent to whatever process you are piping to. Without seeing the full command you are using I can't say exactly why you aren't seeing the STDOUT messages, but they are obviously being swallowed by the redirection. Whether or not that is actually a problem if for you to decide.
the line
$response = <STDIN> ;
causes the script to wait for input from STDIN which is why you see the prompt. You are not piping anything in to STDIN so it waits.
I am having the following Ruby program.
puts "hai"
def mult(a,b)
a * b
end
puts "hello"
def getCostAndMpg
cost = 30000 # some fancy db calls go here
mpg = 30
return cost,mpg
end
AltimaCost, AltimaMpg = getCostAndMpg
puts "AltimaCost = #{AltimaCost}, AltimaMpg = {AltimaMpg}"
I have written a perl script which will extract the functions alone in a Ruby file as follows
while (<DATA>){
print if ( /def/ .. /end/ );
}
Here the <DATA> is reading from the ruby file.
So perl prograam produces the following output.
def mult(a,b)
a * b
end
def getCostAndMpg
cost = 30000 # some fancy db calls go here
mpg = 30
return cost,mpg
end
But, if the function is having block of statements, say for example it is having an if condition testing block means then it is not working. It is taking only up to the "end" of "if" block. And it is not taking up to the "end" of the function. So kindly provide solutions for me.
Example:
def function
if x > 2
puts "x is greater than 2"
elsif x <= 2 and x!=0
puts "x is 1"
else
puts "I can't guess the number"
end #----- My code parsing only up to this
end
Thanks in Advance!
If your code is properly indented, you just want lines that start with def or end, so change your program to:
while (<DATA>){
print if ( /^def/ .. /^end/ );
}
Or run it without a program file at all - run the program from the command line, using -n to have perl treat it as a while loop reading from STDIN:
perl -n -e "print if ( /^def/ .. /^end/ );" < ruby-file.rb
I am not familiar with ruby syntax but if you can ensure good indentation all over the code, you can check based on indentation. Something similar to:
my $add = 0;
my $spaces;
while(my $str = <DATA>) {
if (! $add && $str =~ /^(\s*)def function/) {
$add = 1;
$spaces = $1;
}
if ($add) {
print $_;
$add = 0 if ($str =~ /^$spaces\S/);
}
}
Another option could be counting level of program, something like this:
my $level = 0;
while(<DATA>) {
if(/\b def \b/x .. /\b end \b/x && $level == 0) {
$level++ if /\b if \b/x; # put all statements that closes by end here
$level-- if /\b end \b/x;
print;
}
}
I am not all that familiar with ruby syntax, so you need to put all statements that are closed by end into regex with $level++.
Please note I added \b around those keywords to make sure you are matching whole word and not things like undef as start of function.