How to remove one line from a file using Perl? - perl

I'm trying to remove one line from a text file. Instead, what I have wipes out the entire file. Can someone point out the error?
removeReservation("john");
sub removeTime() {
my $name = shift;
open( FILE, "<times.txt" );
#LINES = <FILE>;
close(FILE);
open( FILE, ">times.txt" );
foreach $LINE (#LINES) {
print NEWLIST $LINE unless ( $LINE =~ m/$name/ );
}
close(FILE);
print("Reservation successfully removed.<br/>");
}
Sample times.txt file:
04/15/2012&08:00:00&bob
04/15/2012&08:00:00&john

perl -ni -e 'print unless /whatever/' filename

Oalder's answer is correct, but he should have tested whether the open statements succeeded or not. If the file times.txt doesn't exist, your program would continue on its merry way without a word of warning that something terrible has happened.
Same program as oalders' but:
Testing the results of the open.
Using the three part open statement which is more goof proof. If your file name begins with > or |, your program will fail with the old two part syntax.
Not using global file handles -- especially in subroutines. File handles are normally global in scope. Imagine if I had a file handle named FILE in my main program, and I was reading it, I called this subroutine. That would cause problems. Use locally scoped file handle names.
Variable names should be in lowercase. Constants are all uppercase. It's just a standard that developed over time. Not following it can cause confusion.
Since oalders put the program in a subroutine, you should pass the name of your file in the subroutine as well...
Here's the program:
#!/usr/bin/env perl
use strict;
use warnings;
removeTime( "john", "times.txt" );
sub removeTime {
my $name = shift;
my $time_file = shift;
if (not defined $time_file) {
#Make sure that the $time_file was passed in too.
die qq(Name of Time file not passed to subroutine "removeTime"\n);
}
# Read file into an array for processing
open( my $read_fh, "<", $time_file )
or die qq(Can't open file "$time_file" for reading: $!\n);
my #file_lines = <$read_fh>;
close( $read_fh );
# Rewrite file with the line removed
open( my $write_fh, ">", $time_file )
or die qq(Can't open file "$time_file" for writing: $!\n);
foreach my $line ( #file_lines ) {
print {$write_fh} $line unless ( $line =~ /$name/ );
}
close( $write_fh );
print( "Reservation successfully removed.<br/>" );
}

It looks like you're printing to a filehandle which you have not yet defined. At least you haven't defined it in your sample code. If you enable strict and warnings, you'll get the following message:
Name "main::NEWLIST" used only once: possible typo at remove.pl line 16.
print NEWLIST $LINE unless ($LINE =~ m/$name/);
This code should work for you:
#!/usr/bin/env perl
use strict;
use warnings;
removeTime( "john" );
sub removeTime {
my $name = shift;
open( FILE, "<times.txt" );
my #LINES = <FILE>;
close( FILE );
open( FILE, ">times.txt" );
foreach my $LINE ( #LINES ) {
print FILE $LINE unless ( $LINE =~ m/$name/ );
}
close( FILE );
print( "Reservation successfully removed.<br/>" );
}
A couple of other things to note:
1) Your sample code calls removeReservation() when you mean removeTime()
2) You don't require the round brackets in your subroutine definition unless your intention is to use prototypes. See my example above.

This is in the FAQ.
How do I change, delete, or insert a line in a file, or append to the beginning of a file?
It's always worth checking the FAQ.

Just in case someone wants to remove all lines from a file.
For example, a file (4th line is empty; 5th line has 3 spaces):
t e st1
test2 a
e
aa
bb bb
test3a
cc
To remove lines which match a pattern some might use:
# Remove all lines with a character 'a'
perl -pi -e 's/.*a.*//' fileTest && sed -i '/^$/d' fileTest;
The result:
t e st1
e
bb bb
cc
Related:
perl -h
# -p assume loop like -n but print line also, like sed
# -i[extension] edit <> files in place (makes backup if extension supplied)
# -e program one line of program (several -e's allowed, omit programfile)
sed -h
# -i[SUFFIX], --in-place[=SUFFIX]
# edit files in place (makes backup if SUFFIX supplied)
Reference 1, Reference 2

Related

How can I know if diamond operator moved to the next file?

I have the following code in a file perl_script.pl:
while (my $line = <>) {
chomp $line;
// etc.
}.
I call the script with more than 1 file e.g.
perl perl_script.pl file1.txt file2.txt
Is there a way to know if the $line is started to read from file2.txt etc?
The $ARGV variable
Contains the name of the current file when reading from <>
and you can save the name and test on every line to see if it changed, updating when it does.
If it is really just about getting to a specific file, as the question seems to say, then it's easier since you can also use #ARGV, which contains command-line arguments, to test directly for the needed name.
One other option is to use eof (the form without parenthesis!) to test for end of file so you'll know that the next file is coming in the next iteration -- so you'll need a flag of some sort as well.
A variation on this is to explicitly close the filehandle at the end of each file so that $. gets reset for each new file, what normally doesn't happen for <>, and then $. == 1 is the first line of a newly opened file
while (<>) {
if ($. == 1) { say "new file: $ARGV" }
}
continue {
close ARGV if eof;
}
A useful trick which is documented in perldoc -f eof is the } continue { close ARGV if eof } idiom on a while (<>) loop. This causes $. (input line number) to be reset between files of the ARGV iteration, meaning that it will always be 1 on the first line of a given file.
There's the eof trick, but good luck explaining that to people. I usually find that I want to do something with the old filename too.
Depending on what you want to do, you can track the filename you're
working on so you can recognize when you change to a new file. That way
you know both names at the same time:
use v5.10;
my %line_count;
my $current_file = $ARGV[0];
while( <> ) {
if( $ARGV ne $current_file ) {
say "Change of file from $current_file to $ARGV";
$current_file = $ARGV;
}
$line_count{$ARGV}++
}
use Data::Dumper;
say Dumper( \%line_count );
Now you see when the file changes, and you can use $ARGV
Change of file from cache-filler.pl to common.pl
Change of file from common.pl to wc.pl
Change of file from wc.pl to wordpress_posts.pl
$VAR1 = {
'cache-filler.pl' => 102,
'common.pl' => 13,
'wordpress_posts.pl' => 214,
'wc.pl' => 15
};
Depending what I'm doing, I might not let the diamond operator do all
the work. This give me a lot more control over what's happening and
how I can respond to things:
foreach my $arg ( #ARGV ) {
next unless open my $fh, '<', $arg;
while( <$fh> ) {
...
}
}

Unable to redirect the output of the system command to a file named error.log and stderr to another file named test_file.errorlog

This perl script is traversing all directories and sub directories, searching for a file named RUN in it. Then it opens the file and runs the 1st line written in the file. The problem is that I am not able to redirect the output of the system command to a file named error.log and STDERR to another file named test_file.errorlog, but no such file is created.
Note that all variable are declared if not found.
find (\&pickup_run,$path_to_search);
### Subroutine for extracting path of directories with RUN FILE PRESENT
sub pickup_run {
if ($File::Find::name =~/RUN/) {
### If RUN file is present , push it into array named run_file_present
push(#run_file_present,$File::Find::name);
}
}
###### Iterate over the array containing paths to directories containing RUN files one by one
foreach my $var (#run_file_present) {
$var =~ s/\//\\/g;
($path_minus_run=$var) =~ s/RUN\b//;
#print "$path_minus_run\n";
my $test_case_name;
($test_case_name=$path_minus_run) =~ s/expression to be replced//g;
chdir "$path_minus_run";
########While iterating over the paths, open each file
open data, "$var";
#####Run the first two lines containing commands
my #lines = <data>;
my $return_code=system (" $lines[0] >error.log 2>test_file.errorlog");
if($return_code) {
print "$test_case_name \t \t FAIL \n";
}
else {
print "$test_case_name \t \t PASS \n";
}
close (data);
}
The problem is almost certainly that $lines[0] has a newline at the end after being read from the file
But there are several improvements you could make
Always use strict and use warnings at the top of every Perl program, and declare all your variables using my as close as possible to their first point of use
Use the three-parameter form of open and always check whether it succeeded, putting the built-in variable $! into your die string to say why it failed. You can also use autodie to save writing the code for this manually for every open, but it requires Perl v5.10.1 or better
You shouldn't put quotes around scalar variables -- just used them as they are. so chdir $path_minus_run and open data, $var are correct
There is also no need to save all the files to be processed and deal with them later. Within the wanted subroutine, File::Find sets you up with $File::Find::dir set to the directory containing the file, and $_ set to the bare file name without a path. It also does a chdir to the directory for you, so the context is ideal for processing the file
use strict;
use warnings;
use v5.10.1;
use autodie;
use File::Find;
my $path_to_search;
find( \&pickup_run, $path_to_search );
sub pickup_run {
return unless -f and $_ eq 'RUN';
my $cmd = do {
open my $fh, '<', $_;
<$fh>;
};
chomp $cmd;
( my $test_name = $File::Find::dir ) =~ s/expression to be replaced//g;
my $retcode = system( "$cmd >error.log 2>test_file.errorlog" );
printf "%s\t\t%s\n", $test_name, $retcode ? 'FAIL' : 'PASS';
}

How do I copy a CSV file, but skip the first line?

I want to write a script that takes a CSV file, deletes its first row and creates a new output csv file.
This is my code:
use Text::CSV_XS;
use strict;
use warnings;
my $csv = Text::CSV_XS->new({sep_char => ','});
my $file = $ARGV[0];
open(my $data, '<', $file) or die "Could not open '$file'\n";
my $csvout = Text::CSV_XS->new({binary => 1, eol => $/});
open my $OUTPUT, '>', "file.csv" or die "Can't able to open file.csv\n";
my $tmp = 0;
while (my $line = <$data>) {
# if ($tmp==0)
# {
# $tmp=1;
# next;
# }
chomp $line;
if ($csv->parse($line)) {
my #fields = $csv->fields();
$csvout->print($OUTPUT, \#fields);
} else {
warn "Line could not be parsed: $line\n";
}
}
On the perl command line I write: c:\test.pl csv.csv and it doesn't create the file.csv output, but when I double click the script it creates a blank CSV file. What am I doing wrong?
Your program isn't ideally written, but I can't tell why it doesn't work if you pass the CSV file on the command line as you have described. Do you get the errors Could not open 'csv.csv' or Can't able to open file.csv? If not then the file must be created in your current directory. Perhaps you are looking in the wrong place?
If all you need to do is to drop the first line then there is no need to use a module to process the CSV data - you can handle it as a simple text file.
If the file is specified on the command line, as in c:\test.pl csv.csv, you can read from it without explicitly opening it using the <> operator.
This program reads the lines from the input file and prints them to the output only if the line counter (the $. variable) isn't equal to one).
use strict;
use warnings;
open my $out, '>', 'file.csv' or die $!;
while (my $line = <>) {
print $out $line unless $. == 1;
}
Yhm.. you don't need any modules for this task, since CSV ( comma separated value ) are simply text files - just open file, and iterate over its lines ( write to output all lines except particular number, e.g. first ). Such task ( skip first line ) is so simple, that it would be probably better to do it with command line one-liner than a dedicated script.
quick search - see e.g. this link for an example, there are numerous tutorials about perl input/output operations
http://learn.perl.org/examples/read_write_file.html
PS. Perl scripts ( programs ) usually are not "compiled" into binary file - they are of course "compiled", but, uhm, on the fly - that's why /usr/bin/perl is called rather "interpreter" than "compiler" like gcc or g++. I guess what you're looking for is some editor with syntax highlighting and other development goods - you probably could try Eclipse with perl plugin for that ( cross platform ).
http://www.eclipse.org/downloads/
http://www.epic-ide.org/download.php/
this
user#localhost:~$ cat blabla.csv | perl -ne 'print $_ if $x++; '
skips first line ( prints out only if variable incremented AFTER each use of it is more than zero )
You are missing your first (and only) argument due to Windows.
I think this question will help you: #ARGV is empty using ActivePerl in Windows 7

Read the last line of file with data in Perl

I have a text file to parse in Perl. I parse it from the start of file and get the data that is needed.
After all that is done I want to read the last line in the file with data. The problem is that the last two lines are blank. So how do I get the last line that holds any data?
If the file is relatively short, just read on from where you finished getting the data, keeping the last non-blank line:
use autodie ':io';
open(my $fh, '<', 'file_to_read.txt');
# get the data that is needed, then:
my $last_non_blank_line;
while (my $line = readline $fh) {
# choose one of the following two lines, depending what you meant
if ( $line =~ /\S/ ) { $last_non_blank_line = $line } # line isn't all whitespace
# if ( line !~ /^$/ ) { $last_non_blank_line = $line } # line has no characters before the newline
}
If the file is longer, or you may have passed the last non-blank line in your initial data gathering step, reopen it and read from the end:
my $backwards = File::ReadBackwards->new( 'file_to_read.txt' );
my $last_non_blank_line;
do {
$last_non_blank_line = $backwards->readline;
} until ! defined $last_non_blank_line || $last_non_blank_line =~ /\S/;
perl -e 'while (<>) { if ($_) {$last = $_;} } print $last;' < my_file.txt
You can use the module File::ReadBackwards in the following way:
use File::ReadBackwards ;
$bw = File::ReadBackwards->new('filepath') or
die "can't read file";
while( defined( $log_line = $bw->readline ) ) {
print $log_line ;
exit 0;
}
If they're blank, just check $log_line for a match with \n;
If the file is small, I would store it in an array and read from the end. If its large, use File::ReadBackwards module.
Here's my variant of command line perl solution:
perl -ne 'END {print $last} $last= $_ if /\S/' file.txt
No one mentioned Path::Tiny. If the file size is relativity small you can do this:
use Path::Tiny;
my $file = path($file_name);
my ($last_line) = $file->lines({count => -1});
CPAN page.
Just remember for the large file, just as #ysth said it's better to use File::ReadBackwards. The difference can be substantial.
sometimes it is more comfortable for me to run shell commands from perl code. so I'd prefer following code to resolve the case:
$result=`tail -n 1 /path/file`;

Can I read and write to multiple filehandles simultaneously (Perl)?

I'm trying to read from two files, and generate output in a third. I first wanted to edit the first one on the go but I didn't find a suitable method save for arrays.
My problem is that the third file (output) is empty whenever I uncomment the "_ref_param_handling" function. BUT the following is what puzzles me the most: If I do a UNIX very basic `cat` system call on the output file at then end (see code below), it works just fine. If I open the filehandle just before and close it right after editing, it also works fine (around my print FILEHANDLE LIST).
I undoubtedly am missing something here. Apart from a problem between my keyboard and my chair, what is it? A filehandle conflict? A scope problem?
Every variable is declared and has the value I want it to have.
Edit (not applicable anymore).
Using IO::File on the three files didn't change anything.
Edit 2 : New full subroutine code
My code works (except when my ref already exists, but that's because of the "append" mode i think) but there might be some mistakes and unperlish ways of coding (sorry, Monks). I, however, use Strict and warnings !
sub _ref_edit($) {
my $manda_def = "$dir/manda_def.list";
my $newrefhandle;
my $ref = $_[0];
(my $refout = $ref) =~ s/empty//;
my $refhandle;
my $parname = '';
my $parvalue = '';
my #val;
_printMan;
my $flush = readline STDIN; # Wait for <enter>
# If one or both of the ref. and the default values are missing
if ( !( -e $manda_def && -e $ref ) ) {
die "Cannot find $ref and/or $manda_def";
}
# Open needed files (ref & default)
open( $refhandle, "<", $ref ) or die "Cannot open ref $ref : $!";
open( $newrefhandle, ">>", $refout )
or die "Cannot open new ref $refout : $!";
# Read each line
while ( my $refline = <$refhandle> ) {
# If line read not an editable macro
if ( $refline =~ /^define\({{(.+)}},\s+{{.*__VALUE__.*}}\)/ ){
$parname = $1; # $1 = parameter name captured in regexp
# Prompt user
$parvalue = _ref_param_handling( $parname, $manda_def );
# Substitution in ref
$refline =~ s/__VALUE__/$parvalue/;
# Param not specified and no default value
$parvalue eq '' ? $refline=~s/__COM__/#/ : $refline=~s/__COM__//;
}
print $newrefhandle $refline;
}
close $newrefhandle;
close $refhandle;
return $refout;
} # End ref edit
the _ref_param_handle subroutine still is :
open( $mde, '<', $_[1] )
or die "Cannot open mandatory/default list $_[1] : $!";
# Read default/mandatory file list
while (<$mde>) {
( $name, $manda, $default, $match, $descript ) = split( /\s+/, $_, 5 );
next if ( $name !~ $ref_param ); # If param read differs from parname
(SOME IF/ELSE)
} # End while <MDE>
close $mde;
return $input;
}
Extract from manda_def file :
NAME Mandatory? Default Match Comm.
PORT y NULL ^\d+$ Database port
PROJECT y NULL \w{1,5} Project name
SERVER y NULL \w+ Server name
modemRouting n NULL .+
modlib y bin .+
modules y sms .+
Extract from ref_file :
define({{PORT}}, {{__VALUE__}})dnl
define({{PROJECT}}, {{__VALUE__}})dnl
define({{SERVER}}, {{__VALUE__}})dnl
define({{modemRouting}}, {{__COM__{{$0}} '__VALUE__'}})dnl
define({{modlib}}, {{__COM__{{$0}} '__VALUE__'}})dnl
define({{modules}}, {{__COM__{{$0}} '__VALUE__'}})dnl
Any help appreciated.
It is unclear what is initialising $refhandle, $newrefhandle and $mde. Depending on the values they have will affect the behaviour of open - i.e. whether it will close any filehandles before opening a new one.
I would suggest that you start using the IO::File interface to open/write to files, as this makes the job of filehandle management much easier, and will avoid any inadvertent closes. Something like...
use IO::File;
my $refhandle = IO::File->new("< $ref") or die "open() - $!";
$refhandle->print(...);
As far as editing files in place goes, this is a common pattern I use to achieve this, make sure of the -i behaviour of perl.
sub edit_file
{
my ($filename) = #_;
# you can re-create the one-liner above by localizing #ARGV as the list of
# files the <> will process, and localizing $^I as the name of the backup file.
local (#ARGV) = ($filename);
local($^I) = '.bak';
while (<>)
{
s/original string/new string/g;
}
continue
{
print;
}
}
try opening the second file handle for input outside the loop and pass a reference to the subroutine _ref_param_handle.Use seek function to seek file back to start.If your file is not too large you can also think of storing the content in an array and the accessing it instead of looping over same contents.
EDIT:
Here is a small example to support what I was trying to say above:
#!/usr/bin/perl -w
sub test
{
my $fh_to_read = $_[0] ;
my $fh_to_write = $_[1] ;
while(<$fh_to_read>)
{
print $fh_to_write $_ ;
}
seek($fh_to_read,0,0) ;
}
open(FH1,"<dummy1");
open(FH2,"<dummy2");
open(FH3,">dummy3");
while(<FH2>)
{
print FH3 "$_" ;
test(\*FH1,\*FH3);
}
Info about perl references
From what I gather, your script wants to convert a file in the following form:
define({{VAR1}}, {{__VALUE__}})
define({{VAR2}}, {{__VALUE__}})
define({{VAR3}}, {{__VALUE__}})
define({{VAR4}}, {{__VALUE__}})
to something like this:
define({{VAR1}}, {{}})
define({{VAR2}}, {{VALUE2}})
define({{VAR3}}, {{VALUE3}})
define({{VAR4}}, {{}})
The following works. I don't know what manda_def means, and also I didn't bother to create an actual variable replacement function.
#!/usr/bin/perl
use strict;
use warnings;
sub work {
my ($ref, $newref, $manda_def) = #_;
# Open needed files (ref & default)
open(my $refhandle, '<', $ref) or die "Cannot open ref $ref : $!";
open(my $newrefhandle, '>', $newref) or die "Cannot open new ref $newref: $!";
# Read each line
while (my $refline = <$refhandle>) {
# if line read is not an editable macro
if ($refline =~ /^define\({{(.+)}},\s+{{.*__VALUE__.*}}\)/){
my $parvalue = _ref_param_handling($1, $manda_def); # manda_def?
# Substitution in ref
$refline =~ s/__VALUE__/$parvalue/;
# Param not specified and no default value
$refline =~ s/__COM__/#/ if $parvalue eq '';
}
print $newrefhandle $refline;
}
close $newrefhandle;
close $refhandle;
return $newref;
}
sub _ref_param_handling {
my %parms = (VAR2 => 'VALUE2', VAR3 => 'VALUE3');
return $parms{$_[0]} if exists $parms{$_[0]};
}
work('ref.txt', 'newref.txt', 'manda.txt');
Guys, I seriously consider hanging myself with my wireless mouse.
My script never failed. I just didn't ran it through the end (it's actually a very long parameter list). The printing is just done as soon as the filehandle is closed (or so I guessed)...
/me *cries*
I've spent 24 hours on this...