Perl. I cannot write on a file. The file stays empty - perl

Question

I wrote the code below.
I cannot find any error in it.
But
say $valid $1;
does not work. $valid file is empty when the program finishes.
What's wrong?
Thanks in advance! ^^
#!/usr/bin/perl
use 5.012;
use strict;
use warnings;
use LWP::Simple;
open my $input, '<', 'c:\perl\015_JiraGet\addrHDP.txt' or die "Cannot read: $!\n";
open my $valid, '<', 'c:\perl\015_JiraGet\valid.txt' or die "Cannot read: $!\n";
my #totalReport;
my $eachAddr;
my $copyFile;
my $copyFilePath = 'c:\perl\015_JiraGet\HADOOP XML\\';
my $tempFile;
my $tempFilePath = 'c:\perl\015_JiraGet\temp.txt';
my $analyzed;
my $analyzedPath = 'c:\perl\015_JiraGet\analyzed - HADOOP.txt';
my $undefCheck;
my $i = 0;
my $j = 0;
my $title = 'temp';
my $dup = 0;
while(<$input>) { chomp; push #totalReport, $_; }
foreach(#totalReport)
{
$eachAddr = $_;
$undefCheck = get($eachAddr);
if(defined($undefCheck) == 0) { next; }
# Copy one XML file to 'temp.txt' and then close the file.
open $tempFile, '>', $tempFilePath or die "Cannot open 1: $!\n";
print $tempFile get($eachAddr);
close $tempFile;
# If the entry is a duplicate, go on to the next entry
open $tempFile, '<', $tempFilePath or die "Cannot open 2: $!\n";
($title, $dup) = isDuplicate($tempFile, $title);
if($dup == 1) { close $tempFile; next; }
close $tempFile;
say ++$i . "th report!!!";
# Copy one XML file to HDD.
if($eachAddr =~ /.*\/(.*)/)
{
say $valid $1;
open $copyFile, '>', $copyFilePath . $1 or die "Cannot open 3: $!\n";
print $copyFile get($eachAddr);
close $copyFile;
}
# If the entry is NOT fixed or resolved, go on to the next entry
open $tempFile, '<', $tempFilePath or die "Cannot open 4: $!\n";
if(isFixCloseResolve($tempFile) == 0) { close $tempFile; next; }
close $tempFile;
# Analyze one entry
open $tempFile, '<', $tempFilePath or die "Cannot open 5: $!\n";
open $analyzed, '>>', $analyzedPath or die "Cannot open 6: $!\n";
analyzeOneReport($tempFile, $analyzed);
close $tempFile;
close $analyzed;
say ' ' . ++$j . "th fixed & closed report!!!";
}
say "$i total reports.";
say "$j total fixed & closed reports.";
close $input;
close $valid;
say "Finished!";
sub isDuplicate
{
my $iReport = $_[0];
my $title = 'temp';
my $dup = 0;
while(<$iReport>)
{
if ($_ =~ /.*\<title>(.*)\<\/title>/)
{
if($1 ne 'ASF JIRA') { $title = $1; if($title eq $_[1]) { $dup = 1; } last; }
}
}
return ($title, $dup);
}
# returns 1 if an entry is a Bug and Fixed and Closed
sub isFixCloseResolve
{
my $iReport = $_[0];
my $isCloseResolve = 0;
my $isFixed = 0;
while(<$iReport>)
{
if ($_ =~ /.*\<status[^>]*>(.*)\<\/status>/) { if(($1 eq 'Closed')||($1 eq 'Resolved')) { $isCloseResolve = 1;} }
elsif($_ =~ /.*\<resolution[^>]*>(.*)\<\/resolution>/) { if($1 eq 'Fixed') { $isFixed = 1;} }
}
return $isCloseResolve * $isFixed;
}
sub analyzeOneReport
{
my $iReport = $_[0];
my $oReport = $_[1];
while(<$iReport>)
{
chomp;
if ($_ =~ /.*\<title>(.*)\<\/title>/) { if($1 ne 'ASF JIRA') { say $oReport "Title : $1"; } }
elsif($_ =~ /.*\<assignee username="(.*)">.*\<\/assignee>/) { say $oReport "Assignee: $1"; }
elsif($_ =~ /.*\<reporter username="(.*)">.*\<\/reporter>/) { say $oReport "Reporter: $1"; }
elsif($_ =~ /.*\<type[^>]*>(.*)\<\/type>/) { say $oReport "Type : $1"; }
elsif($_ =~ /.*\<priority[^>]*>(.*)\<\/priority>/) { say $oReport "Priority: $1"; }
elsif($_ =~ /.*\<created>(.*)\<\/created>/) { say $oReport "Created : $1"; }
elsif($_ =~ /.*\<resolved>(.*)\<\/resolved>/) { say $oReport "Resolved: $1"; }
}
say $oReport '--------------------------------------------';
}
--- Postscript ---
Oh, I was wrong on '>' part!! Thank you everyone!!
But when I changed that into '>', still nothing was written on the file 'DURING PROGRAM RUNNING TIME'.
So I was confused...and I found that Perl actually writes the contents to the file 'WHEN IT CLOSED THE FILE'.
So during running time, for 4~8 hours, I could not see anything in the file.
Data is written on the file when the file is closed.
That's one of the reason why I thought this code was not working. ^^;
Hope nobody else suffer from this problem again! :)

Answer 1

Here:
open my $valid, '<',....
you have opened $valid for reading. If you wish to write to the file, you must instead write:
open my $valid, '>',....
If you need to keep existing contents and write only to the end then instead use
open my $valid, '>>',....

Answer 2

You're only opening the file under the $valid file handle with read capabilities, as you can see from this line:
open my $valid, '<', 'c:\perl\015_JiraGet\valid.txt' or die "Cannot read: $!\n";
So nothing you write to the file will actually go into it. Change it to read/write (or append, if you need it, just use +>> instead of +> in the code below), and you should be good, as follows:
open my $valid, '+>', 'c:\perl\015_JiraGet\valid.txt' or die "Cannot read: $!\n";

Answer 3

I am going to review this code as if it had been posted to Code Review.
First off you are writing in Perl as if it were C. Which in general is not that bad, but it does mean that you are doing quite a bit more work than is necessary.
Instead of using this overly verbose, and potentially memory intensive:
my #totalReport
...
while(<$input>) { chomp; push #totalReport, $_; }
foreach(#totalReport)
{
$eachAddr = $_;
...
}
while( my $addr = <$input> ){
chomp $addr;
...
}
Notice how I've eliminated a variable, and made it so that it loops on the input once, instead of twice. It also doesn't keep the values in memory over the entire length of the program.
Instead of opening a file for writing, closing it, and opening it again:
my $tempFile;
open $tempFile, '>', $tempFilePath or die "Cannot open 1: $!\n";
print $tempFile get($eachAddr);
close $tempFile;
open $tempFile, '<', $tempFilePath or die "Cannot open 2: $!\n";
open my $tempFile, '+>', $tempFilePath or die "Can't open '$tempFilePath' with mode'+>': '$!'";
print $tempFile get($eachAddr);
seek $tempFile, 0, 0;
Instead of getting the text at the given URL twice, and using a weird defined test:
$undefCheck = get($eachAddr);
if(defined($undefCheck) == 0) { next; }
...
print $tempFile get($eachAddr);
my $text = get( $addr );
next unless defined $text;
...
print $tempFile $text;
Instead of a bunch of:
open ... or die ...;
I would use autodie.
use autodie;
...
# will now die on error and will tell you the file it fails on.
open my $fh, '<', $filename;
Another thing I would like to point out is that die "...\n" prevents die from appending the location of the error. The only time you should do that is if the default behaviour is unhelpful.
If you closed $tempFile before checking $dup this could be simpler:
if($dup == 1) { close $tempFile; next; }
close $tempFile;
close $tempFile;
next if $dup;
Instead of this repetitive block of code:
while(<$iReport>)
{
chomp;
if ($_ =~ /.*\<title>(.*)\<\/title>/) { if($1 ne 'ASF JIRA') { say $oReport "Title : $1"; } }
elsif($_ =~ /.*\<assignee username="(.*)">.*\<\/assignee>/) { say $oReport "Assignee: $1"; }
elsif($_ =~ /.*\<reporter username="(.*)">.*\<\/reporter>/) { say $oReport "Reporter: $1"; }
elsif($_ =~ /.*\<type[^>]*>(.*)\<\/type>/) { say $oReport "Type : $1"; }
elsif($_ =~ /.*\<priority[^>]*>(.*)\<\/priority>/) { say $oReport "Priority: $1"; }
elsif($_ =~ /.*\<created>(.*)\<\/created>/) { say $oReport "Created : $1"; }
elsif($_ =~ /.*\<resolved>(.*)\<\/resolved>/) { say $oReport "Resolved: $1"; }
}
use List::Util qw'max';
my #simple_tags = qw'title type priority created resolved';
my $simple_tags_length = max map length, #simple_tags, qw'assignee reporter';
my $simple_tags = join '|', #simple_tags;
...
while( <$iReport> ){
my($tag,$contents);
if( ($tag,$contents) = /<($simple_tags)[^>]*>(.*?)<\/\g{1}>/ ){
}elsif( ($tag,$contents) = /<(assignee|reporter) username="(.*?)">.*?<\/\g{1}>/ ){
}else{ next }
printf $oReport "%-${simple_tags_length}s: %s\n", ucfirst($tag), $contents;
}
While this code isn't any shorter, or clearer, it would be very easy to add another tag to compare against. So it isn't really better, as less repetitive.
I would like to point out that $_ =~ /.../ is better written as /.../.
You could use or instead of if/elsif/else with empty blocks.
...
while( <$iReport> ){
/<($simple_tags)[^>]*>(.*?)<\/\g{1}>/
or /<(assignee|reporter) username="(.*?)">.*?<\/\g{1}>/
or next;
my($tag,$contents) = ($1,$2);
printf $oReport "%-${simple_tags_length}s: %s\n", ucfirst($tag), $contents;
}
It might be best to combine them into a single regex using /x and (?<NAME>REGEX) syntax with %- or %+.
...
while( <$iReport> ){
/
(?:
# simple tags
< (?<tag> $simple_tags ) [^>]* >
# contents between tags
(?<contents> .*? )
|
# tags with contents in `username` attribute
<
(?<tag> assignee|reporter )
[ ]
# contents in `username` attribute
username = "(?<contents> .*? )"
>
.*? # throw out stuff between tags
)
<\/ \g{tag} > # end tag matches start tag
/x or next; # skip if it doesn't match
printf $oReport "%-${simple_tags_length}s: %s\n", ucfirst($+{tag}), $+{contents};
}
Or even use (DEFINE) (I'll leave that as an exercise for the reader since this is already fairly long).
Perhaps the worst part of the code is that you define almost all of you variables up-front.
my #totalReport;
my $eachAddr;
my $copyFile;
my $copyFilePath = 'c:\perl\015_JiraGet\HADOOP XML\\';
my $tempFile;
my $tempFilePath = 'c:\perl\015_JiraGet\temp.txt';
my $analyzed;
my $analyzedPath = 'c:\perl\015_JiraGet\analyzed - HADOOP.txt';
my $undefCheck;
my $i = 0;
my $j = 0;
my $title = 'temp';
my $dup = 0;
This means that you are practically using global variables. While some of these appear to need to be defined there, some of them don't, and therefore shouldn't be defined there. You should really be defining your variables at the point you need them, or at least at the beginning of the block where you need them.
The reason you aren't getting the output until the file is closed is because Perl buffers the output.
Perl normally buffers output so it doesn't make a system call for every bit of output. By saving up output, it makes fewer expensive system calls. …
- perlfaq5
The old way to turn off buffering is to select the file for output and set $| to a non-zero value, and then re-select the original output.
{
my $previous_default = select($file); # save previous default output handle
$| = 1; # autoflush
select($previous_default); # restore previous default output handle
}
The new way is to use $file->autoflush which comes from IO::Handle.
(The module will get automatically loaded for you on recent versions of Perl 5)
You can also flush the output when you choose by using flush or $file->flush.
IO::Handle also adds a $file->printflush which turns on autoflush temporarily during the print.